Evaluating methods of outlier detection when benchmarking clinical registry data–a simulation study

Ahern, S., Hopper, I., Evans, S.M.: Clinical quality registries for clinician-level reporting: strengths and limitations. Med. J. Aust. 206(10), 427–429 (2017)

Article  PubMed  Google Scholar 

Austin, P.C.: A comparison of bayesian methods for profiling hospital performance. Med. Decis. Making 22(2), 163–172 (2002). https://doi.org/10.1177/0272989X0202200213

Article  PubMed  Google Scholar 

Austin, P.C., Naylor, C.D., Tu, J.V.: A comparison of a Bayesian vs a frequentist method for profiling hospital performance. J. Eval. Clin. Pract. 7(1), 35–45 (2001). https://doi.org/10.1046/j.1365-2753.2001.00261.x

Article  CAS  PubMed  Google Scholar 

Australian Commission on Safety and Quality in Healthcare (ACSQHC): Australian Register of Clinical Registries. https://www.safetyandquality.gov.au/publications-and-resources/australian-register-clinical-registries (2022). Accessed 1 November 2022.

Behrendt, K., Groene, O.: Mechanisms and effects of public reporting of surgeon outcomes: a systematic review of the literature. Health Policy 120(10), 1151–1161 (2016). https://doi.org/10.1016/j.healthpol.2016.08.003

Article  PubMed  Google Scholar 

Blackmore, A.R., Leonard, J., Madayag, R., Bourg, P.W.: Using the trauma quality improvement program metrics data to enhance clinical practice. J. Trauma Nurs. 26(3), 121–127 (2019). https://doi.org/10.1097/JTN.0000000000000436

Article  PubMed  Google Scholar 

Blumenthal, S.: The use of clinical registries in the United States: a landscape survey. eGEMS 5(1), 26–26 (2017). https://doi.org/10.5334/egems.248

Article  PubMed  PubMed Central  Google Scholar 

Brown, W.A., Ahern, S., MacCormick, A.D., Reilly, J.R., Smith, J.A., Watters, D.A.: Clinical quality registries: urgent reform is required to enable best practice and best care. ANZ J. Surg. 92(1–2), 23–26 (2022). https://doi.org/10.1111/ans.17438

Article  PubMed  Google Scholar 

Dimick, J.B., Ghaferi, A.A., Osborne, N.H., Ko, C.Y., Hall, B.L.: Reliability adjustment for reporting hospital outcomes with surgery. Ann. Surg. 255(4), 703–707 (2012). https://doi.org/10.1097/SLA.0b013e31824b46ff

Article  PubMed  Google Scholar 

Eijkenaar, F., van Vliet, R.C.J.A.: Performance profiling in primary care: does the choice of statistical model matter? Med. Decis. Making 34(2), 192–205 (2014). https://doi.org/10.1177/0272989X13498825

Article  PubMed  Google Scholar 

Endo, H., Ichihara, N., Miyata, H., Uchino, S., Hashimoto, S., Aoki, Y., Hashiba, E., Hatakeyama, J., Hayakawa, K., Irie, H., Kawasaki, T., Kumasawa, J., Kurosawa, H., Nakamura, T., Ohbe, H., Okamoto, H., Shigemitsu, H., Tagami, T., Takaki, S., Takimoto, K., Uchida, M.: Development and validation of the predictive risk of death model for adult patients admitted to intensive care units in Japan: an approach to improve the accuracy of healthcare quality measures. J. Intensive Care 9(1), 18 (2021). https://doi.org/10.1186/s40560-021-00533-z

Article  PubMed  PubMed Central  Google Scholar 

Evans, S.M., Bohensky, M., Cameron, P.A., McNeil, J.: A survey of Australian clinical registries: can quality of care be measured? Intern. Med. J. 41(1a), 42–48 (2011a). https://doi.org/10.1111/j.1445-5994.2009.02068.x

Article  CAS  PubMed  Google Scholar 

Evans, S.M., Scott, I.A., Johnson, N.P., Cameron, P.A., McNeil, J.J.: Development of clinical-quality registries in Australia: the way forward. Med. J. Aust. 194(7), 360–363 (2011b). https://doi.org/10.5694/j.1326-5377.2011.tb03007.x

Article  PubMed  Google Scholar 

Guglielmi, A., Ieva, F., Paganoni, A.M., Ruggeri, F., Soriano, J.: Semiparametric Bayesian models for clustering and classification in the presence of unbalanced in-hospital survival. J. r. Stat. Soc. C 63(1), 25–46 (2014). https://doi.org/10.1111/rssc.12021

Article  Google Scholar 

Hamilton, B.H., Hall, B.L., Huffman, K.M., Zhou, L., Richards, K.E., Cohen, M.E., Paruch, J.L., Ko, C.Y.: Profiling individual surgeon performance using information from a high-quality clinical registry: opportunities and limitations. J. Am. Coll. Surg. 221(5), 901–913 (2015). https://doi.org/10.1016/j.jamcollsurg.2015.07.454

Article  PubMed  Google Scholar 

Hansen, J., Ahern, S., Earnest, A.: Evaluations of statistical methods for outlier detection when benchmarking in clinical registries: a systematic review. BMJ Open 13(7), e069130 (2023). https://doi.org/10.1136/bmjopen-2022-069130

Article  PubMed  PubMed Central  Google Scholar 

Hess, C.N., Rao, S.V., McCoy, L.A., Neely, M.L., Peterson, E.D., Singh, M., Spertus, J.A., Krone, R.J., Weaver, W.D.: Identification of hospital outliers in bleeding complications after percutaneous coronary intervention. Circ. Cardiovasc. Qual. Outcomes 8(1), 15–22 (2015). https://doi.org/10.1161/CIRCOUTCOMES.113.000749

Article  PubMed  Google Scholar 

Hoque, D.M.E., Kumari, V., Hoque, M., Ruseckaite, R., Romero, L., Evans, S.M.: Impact of clinical registries on quality of patient care and clinical outcomes: a systematic review. PLoS One 12(9), e0183667 (2017). https://doi.org/10.1371/journal.pone.0183667

Article  CAS  PubMed  PubMed Central  Google Scholar 

Ieva, F., Paganoni, A.M.: Detecting and visualizing outliers in provider profiling via funnel plots and mixed effect models. Health Care Manag. Sci. 18(2), 166–172 (2015). https://doi.org/10.1007/s10729-013-9264-9

Article  PubMed  Google Scholar 

Jones, H.E., Ohlssen, D.I., Spiegelhalter, D.J.: Use of the false discovery rate when comparing multiple health care providers. J. Clin. Epidemiol. 61(3), 232–240 (2008). https://doi.org/10.1016/j.jclinepi.2007.04.017

Article  PubMed  Google Scholar 

Kalbfleisch, J.D., Wolfe, R.A.: On monitoring outcomes of medical providers. Stat. Biosci. 5(2), 286–302 (2013). https://doi.org/10.1007/s12561-013-9093-x

Article  Google Scholar 

Kasza, J., Moran, J.L., Solomon, P.J.: Evaluating the performance of Australian and New Zealand intensive care units in 2009 and 2010. Stat. Med. 32(21), 3720–3736 (2013). https://doi.org/10.1002/sim.5779

Article  CAS  PubMed  Google Scholar 

Kasza, J., Polkinghorne, K.R., Wolfe, R., McDonald, S.P., Marshall, M.R.: Comparing dialysis centre mortality outcomes across Australia and New Zealand: identifying unusually performing centres 2008–2013. BMC Health Serv. Res. 18, 1007 (2018). https://doi.org/10.1186/s12913-018-3832-0

Article  PubMed  PubMed Central  Google Scholar 

Lecky, F., Woodford, M., Edwards, A., Bouamra, O., Coats, T.: Trauma scoring systems and databases. Br. J. Anaesth. 113(2), 286–294 (2014). https://doi.org/10.1093/bja/aeu242

Article  CAS  PubMed  Google Scholar 

Longford, N.T.: Decision theory for comparing institutions. Stat. Med. 37(3), 457–472 (2018). https://doi.org/10.1002/sim.7525

Article  PubMed  Google Scholar 

MacKenzie, T.A.P., Grunkemeier, G.L.P., Grunwald, G.K.P., O’Malley, A.J.P., Bohn, C.M.S., Wu, Y.P., Malenka, D.J.M.D.: A primer on using shrinkage to compare in-hospital mortality between centers. Ann. Thorac. Surg. 99(3), 757–761 (2015). https://doi.org/10.1016/j.athoracsur.2014.11.039

Article  PubMed  Google Scholar 

Manktelow, B.N., Seaton, S.E., Evans, T.A.: Funnel plot control limits to identify poorly performing healthcare providers when there is uncertainty in the value of the benchmark. Stat. Methods Med. Res. 25(6), 2670–2684 (2016). https://doi.org/10.1177/0962280214530281

Article  PubMed  Google Scholar 

McNeil, J.J., Evans, S.M., Johnson, N.P., Cameron, P.A.: Clinical-quality registries: their role in quality improvement. Med. J. Aust. 192(5), 244–245 (2010). https://doi.org/10.5694/j.1326-5377.2010.tb03499.x

Article  PubMed  Google Scholar 

Moore, L., Hanley, J.A., Turgeon, A.F., Lavoie, A.: Evaluating the performance of trauma centers: hierarchical modeling should be used. J. Trauma 69(5), 1132–1137 (2010). https://doi.org/10.1097/TA.0b013e3181cc8449

Article  PubMed  Google Scholar 

Moran, J.L., Solomon, P.J.: Fixed effects modelling for provider mortality outcomes: analysis of the Australia and New Zealand Intensive Care Society (ANZICS) adult patient data-base. PLoS One 9(7), e102297 (2014). https://doi.org/10.1371/journal.pone.0102297

Article  CAS  PubMed  PubMed Central  Google Scholar 

Morris, T.P., White, I.R., Crowther, M.J.: Using simulation studies to evaluate statistical methods. Stat. Med. 38(11), 2074–2102 (2019). https://doi.org/10.1002/sim.8086

Comments (0)

No login
gif