External Validation of Cochlear Implant Screening Tools Demonstrates Modest Generalizability

Abstract

ObjectiveTo assess the clinical application of five recently published cochlear implant (CI) candidacy evaluation (CICE) referral screening tools through external validation.Study DesignRetrospective cohort study.SettingTertiary otology/neurotology practice.PatientsAdults who underwent CICE between December 2020 and September 2021.Intervention(s)CICE referral screening tools versus CI candidacy criteria.Main Outcome Measure(s)CICE screening tool performance, based on the ability to identify patients who met the CI candidacy criteria, was evaluated. CI candidacy criteria were defined as best-aided AzBio sentences at +10 signal-to-noise ratio and either 60% or less accuracy to reflect traditional criteria used in clinical settings or 40% or less accuracy (only patients 65 years or older) to reflect Medicare-eligible criteria.ResultsScreening criteria of proposed CICE referral tools vary widely across pure-tone average and word recognition scores. When screened by traditional criteria, the sensitivities and specificities of these referral tools varied from 40 to 77% and from 22 to 86%, respectively. When screened by Medicare-eligible criteria, sensitivities and specificities varied from 41 to 81% and from 24 to 91%, respectively. The screening tool proposed by Zwolan et al. (Otol Neurotol 2020;41(7):895–900) demonstrated the best overall performance for traditional (Youden's J, 0.37; sensitivity, 62%; specificity, 75%) and Medicare-eligible patients (Youden's J, 0.44; sensitivity, 66%; specificity, 78%). All screening tools performed worse on the validation cohort compared with their respective development cohorts.ConclusionsCurrent tools for determining CICE referral have diverse screening criteria. These combinations of pure-tone average and word recognition score are modestly successful at identifying CI candidates.

Machine Learning Approach for Screening Cochlear Implant Candidates: Comparing With the 60/60 Guideline

“…Other referral guidelines and candidacy calculators are solely based on audiometry and for patients who would primarily be eligible under Medicare and traditional criteria (23)(24)(25)(26)(27)(28)(29). Assessment of five CICE referral tools demonstrated best performance in balancing sensitivity and specificity by the 60/60 guideline, although all screening tools performed worse on a validation cohort (30). More recent screening tools, which also only incorporate audiometry, used smaller cohorts of 132 (25) and 252 (24) patients with no cross-or external-validation of the proposed guidelines.…”

“…Primary end points for the RF model were sensitivity, specificity, and accuracy, which were all compared with those of the 60/60 guideline, the best performer among commonly used referral guidelines (30). Moreover, the importance of each input variable's influence on candidacy was measured by the mean decrease in Gini coefficient (MDG).…”

“…Recently, multiple screening tools have been developed using data from standard audiograms to assist hearing professionals with their clinical decision making (23)(24)(25)(26)(27)(28)(29). These tools have inconsistent sensitivity and specificity, continue to propose complex guidelines, and are only modestly successful at identifying potential CI candidates with an external data set (30). The present study aimed to develop a machine learning-based referral guideline for patients undergoing CICE using the largest cohort of patients to date.…”

“…The present study aimed to develop a machine learning-based referral guideline for patients undergoing CICE using the largest cohort of patients to date. A secondary objective was to compare the algorithm's performance with the widely used "60/60 guideline," which has demonstrated the best balance of sensitivity and specificity among the major proposed screening criteria (30).…”

Responsible Imputation of Missing Speech Perception Testing Data & Analysis of 4,739 Observations and Predictors of Performance

“…Screening tools, based on audiograms and word recognition scores, have been developed to assist hearing professionals decide whether to refer for additional evaluation (6–10). However, external validation of these screening tools demonstrates modest generalizability to other cohorts (11). Further development and validation in large cohorts will improve their generalizability and streamline appropriate referral to CI candidacy evaluation.…”