Comparative testing of reliability and audit utility of ordinal objective calculus complexity scores. Can we make an informed choice yet?

Abstract

Objectives To assess the reliability of the Guy's Stone Score, the Seoul National University Renal Stone Complexity (S‐ReSC) score and the S.T.O.N.E. scores in percutaneous nephrolithotomy (PCNL), and assess their utility in discriminating outcomes [stone free rate (SFR), complications, need for multiple PCNL sessions, and auxiliary procedures] valid across parameters of experience of surgeon, independence from surgical approach, and variations in institution‐specific instrumentation. Patients and methods A prospectively maintained database of two tertiary institutions was analysed (606 cases). Institutes differed in instrumentation, while the overall surgical team comprised: two trainees (experience <100 cases), two junior consultants (experience 100–200 cases), and two senior surgeons (experience >1000 cases). Scores were assigned and re‐assigned after 4 months by one trainee and an expert surgeon. Inter‐rater and test‐retest agreement were analysed by Cohen's κ and intraclass correlation coefficient. Multivariate logistic regression models were created adjusting outcomes for the institution, comorbidity, Amplatz size, access tract location, the number of punctures, the experience level of the surgeon, and individual scoring system, and receiver operating curves were analysed for comparison. Results Despite some areas of inconsistencies, individually all scores had excellent inter‐rater and test‐retest concordance. On multivariable analyses, while the experience of the surgeon and surgical approach characteristics (such as access tract location, Amplatz size, and number of punctures) remained independently associated with different outcomes in varying combinations, calculus complexity scores were found consistently to be independently associated with all outcomes. The S‐ReSC score had a superior association with SFR, the need for multiple PCNL sessions, and auxiliary procedures. Conclusion Individually all scoring systems performed well. On cross comparison, the S‐ReSC score consistently emerged to be more superiorly associated with all outcomes, signifying the importance of the distributional complexity of the calculus (which also indirectly amalgamates the influence of stone number, size, and anatomical location) in discriminating outcomes. Our study proves the utility of scoring systems in prognosticating multiple outcomes and also clarifies important aspects of their practical application including future roles such as benchmarking, audit, training, and objective assessment of surgical technique modifications.

Stepwise case selection using Guy's stone score reduces complications during percutaneous nephrolithotomy training

“…Benchmarking of trainers/experts across ours and reference hospital has been done by us in a separate study and not further discussed here. [14] Additional multivariable analyses of SFR and complications was done while adjusting for Charlson Comorbidity Index Score, body mass index, the Amplatz size, access tract location (whether infracostal or supracostal), the number of punctures, calculus complexity defined by GSS, and type of training (stepwise vs. traditional).…”