A Causal Deep Learning Framework for Classifying Phonemes in Cochlear Implants

Chu, Kevin M.; Collins, Leslie M.; Mainsah, Boyla O.

doi:10.1109/icassp39728.2021.9413986

Cited by 4 publications

(2 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent years, some CI sound processing approaches have shifted from traditional to deep learning based solutions. Neural networks used for CI applications include the feed-forward neural network [20], [21], deep neural network (DNN) [22], [23], convolutional neural network (CNN) [24], fully convolutional network (FCN) [25], [26], recurrent neural network (RNN) based on the long short-term memory (LSTM) architecture [27], [28], and deep recurrent neural network (DRNN) [29]. Among the aforementioned studies, deep learning based speech enhancement approaches outperform traditional ones in both simulations and CI recipients [20]- [24].…”

Section: Introductionmentioning

confidence: 99%

ElectrodeNet – A Deep Learning Based Sound Coding Strategy for Cochlear Implants

Huang¹,

Chao²,

Yu³

et al. 2022

Preprint

View full text Add to dashboard Cite

<p>ElectrodeNet, a deep-learning based sound coding strategy for the cochlear implant (CI), is proposed in this study. </p> <p>ElectrodeNet emulates the ACE strategy by replacing the conventional envelope detection using various artificial neural networks, and the extended ElectrodeNet-CS strategy further incorporates the channel selection (CS) in the network. Network models of deep neural network (DNN), convolutional neural network (CNN), and long short-term memory (LSTM) were trained using the Fast Fourier Transformed bins and electrode stimulation patterns from the processing of the ACE strategy for clean speech. Objective speech understanding using short-time objective intelligibility (STOI) and normalized covariance metric (NCM) was estimated for ElectrodeNet with the factors of network architecture, dataset language, and noise type using CI simulations. Subjective listening tests for vocoded Mandarin speech were conducted with normal-hearing listeners to measure sentence recognition scores. DNN, CNN, and LSTM based ElectrodeNets exhibited strong correlations to ACE in objective and subjective scores using mean squared error (MSE), linear correlation coefficient (LCC) and Spearman’s rank correlation coefficient (SRCC). The ElectrodeNet-CS strategy was capable of producing N-of-M compatible electrode patterns using a modified DNN network to embed maxima selection, and to perform in similar or even slightly above average in STOI and sentence recognition compared to ACE. The methods and findings in this study demonstrated the feasibility and potential of using deep learning in CI sound coding strategy.</p>

show abstract

Section: Introductionmentioning

confidence: 99%

ElectrodeNet – A Deep Learning Based Sound Coding Strategy for Cochlear Implants

Huang¹,

Chao²,

Yu³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In recent years, some CI sound processing approaches have started to shift from traditional to deep learning based solutions. Distinctive types of network architectures, including deep (feedforward) neural network (DNN) [23]- [26], convolutional neural network (CNN) [27]- [29], and recurrent neural network (RNN) [30]- [32], have been investigated for the CI. Among the aforementioned studies, deep learning based speech enhancement approaches outperform traditional ones in both simulations and CI recipients [23]- [27].…”

mentioning

confidence: 99%

ElectrodeNet—A Deep-Learning-Based Sound Coding Strategy for Cochlear Implants

Huang

Chao

Tsao

2024

IEEE Trans. Cogn. Dev. Syst.

View full text Add to dashboard Cite

ElectrodeNet, a deep learning based sound coding strategy for the cochlear implant (CI), is proposed to emulate the advanced combination encoder (ACE) strategy by replacing the conventional envelope detection using various artificial neural networks. The extended ElectrodeNet-CS strategy further incorporates the channel selection (CS). Network models of deep neural network (DNN), convolutional neural network (CNN), and long short-term memory (LSTM) were trained using the Fast Fourier Transformed bins and channel envelopes obtained from the processing of clean speech by the ACE strategy. Objective speech understanding using short-time objective intelligibility (STOI) and normalized covariance metric (NCM) was estimated for ElectrodeNet using CI simulations. Sentence recognition tests for vocoded Mandarin speech were conducted with normalhearing listeners. DNN, CNN, and LSTM based ElectrodeNets exhibited strong correlations to ACE in objective and subjective scores using mean squared error (MSE), linear correlation coefficient (LCC) and Spearman's rank correlation coefficient (SRCC). The ElectrodeNet-CS strategy was capable of producing N-of-M compatible electrode patterns using a modified DNN network to embed maxima selection, and to perform in similar or even slightly higher average in STOI and sentence recognition compared to ACE. The methods and findings demonstrated the feasibility and potential of using deep learning in CI coding strategy.

show abstract

A computational model to simulate spectral modulation and speech perception experiments of cochlear implant users

Alvarez

Kipping²,

Nogueira³

2023

Front. Neuroinform.

View full text Add to dashboard Cite

Speech understanding in cochlear implant (CI) users presents large intersubject variability that may be related to different aspects of the peripheral auditory system, such as the electrode–nerve interface and neural health conditions. This variability makes it more challenging to proof differences in performance between different CI sound coding strategies in regular clinical studies, nevertheless, computational models can be helpful to assess the speech performance of CI users in an environment where all these physiological aspects can be controlled. In this study, differences in performance between three variants of the HiRes Fidelity 120 (F120) sound coding strategy are studied with a computational model. The computational model consists of (i) a processing stage with the sound coding strategy, (ii) a three-dimensional electrode-nerve interface that accounts for auditory nerve fiber (ANF) degeneration, (iii) a population of phenomenological ANF models, and (iv) a feature extractor algorithm to obtain the internal representation (IR) of the neural activity. As the back-end, the simulation framework for auditory discrimination experiments (FADE) was chosen. Two experiments relevant to speech understanding were performed: one related to spectral modulation threshold (SMT), and the other one related to speech reception threshold (SRT). These experiments included three different neural health conditions (healthy ANFs, and moderate and severe ANF degeneration). The F120 was configured to use sequential stimulation (F120-S), and simultaneous stimulation with two (F120-P) and three (F120-T) simultaneously active channels. Simultaneous stimulation causes electric interaction that smears the spectrotemporal information transmitted to the ANFs, and it has been hypothesized to lead to even worse information transmission in poor neural health conditions. In general, worse neural health conditions led to worse predicted performance; nevertheless, the detriment was small compared to clinical data. Results in SRT experiments indicated that performance with simultaneous stimulation, especially F120-T, were more affected by neural degeneration than with sequential stimulation. Results in SMT experiments showed no significant difference in performance. Although the proposed model in its current state is able to perform SMT and SRT experiments, it is not reliable to predict real CI users' performance yet. Nevertheless, improvements related to the ANF model, feature extraction, and predictor algorithm are discussed.

show abstract

A Causal Deep Learning Framework for Classifying Phonemes in Cochlear Implants

Cited by 4 publications

References 26 publications

ElectrodeNet – A Deep Learning Based Sound Coding Strategy for Cochlear Implants

ElectrodeNet – A Deep Learning Based Sound Coding Strategy for Cochlear Implants

ElectrodeNet—A Deep-Learning-Based Sound Coding Strategy for Cochlear Implants

A computational model to simulate spectral modulation and speech perception experiments of cochlear implant users

Contact Info

Product

Resources

About