ElectrodeNet – A Deep Learning Based Sound Coding Strategy for Cochlear Implants

Abstract

<p>ElectrodeNet, a deep-learning based sound coding strategy for the cochlear implant (CI), is proposed in this study. </p> <p>ElectrodeNet emulates the ACE strategy by replacing the conventional envelope detection using various artificial neural networks, and the extended ElectrodeNet-CS strategy further incorporates the channel selection (CS) in the network. Network models of deep neural network (DNN), convolutional neural network (CNN), and long short-term memory (LSTM) were trained using the Fast Fourier Transformed bins and electrode stimulation patterns from the processing of the ACE strategy for clean speech. Objective speech understanding using short-time objective intelligibility (STOI) and normalized covariance metric (NCM) was estimated for ElectrodeNet with the factors of network architecture, dataset language, and noise type using CI simulations. Subjective listening tests for vocoded Mandarin speech were conducted with normal-hearing listeners to measure sentence recognition scores. DNN, CNN, and LSTM based ElectrodeNets exhibited strong correlations to ACE in objective and subjective scores using mean squared error (MSE), linear correlation coefficient (LCC) and Spearman’s rank correlation coefficient (SRCC). The ElectrodeNet-CS strategy was capable of producing N-of-M compatible electrode patterns using a modified DNN network to embed maxima selection, and to perform in similar or even slightly above average in STOI and sentence recognition compared to ACE. The methods and findings in this study demonstrated the feasibility and potential of using deep learning in CI sound coding strategy.</p>

ElectrodeNet – A Deep Learning Based Sound Coding Strategy for Cochlear Implants

“…Each subject listened to the simulated CI sound simultaneously played to both ears via Telephonics TDH-39P headphones in a sound isolation room. The experiment was carried out with NCU-CI, a MATLAB-based CI experiment platform with a graphical user interface in traditional Chinese [44], [48], [82]. In the practice session, each subject was instructed on the experimental procedure by being familiarized with sentences from List 16 of the TMHINT material presented at a fixed and comfortable sound level.…”

“…Furthermore, the core signal processing techniques of traditional coding strategies, envelope detection (extraction) and channel selection (CS), are not differentiable, and hence unable to be directly combined and optimized with other neural network based front-end and post-processing modules in a joint-training and end-to-end learning manner [45]- [47]. Consequently, ElectrodeNet was proposed to demonstrate the concepts of deep learning based CI coding strategy using DNN, CNN, and long short-term memory (LSTM) network models to imitate the envelope detection of the ACE strategy [48]. A comprehensive study was carried out for the performance correlation between ElectrodeNet and ACE using objective evaluation with vocoded sentences for the factors of network architecture, dataset language, and noise type.…”