DNN-based performance measures for predicting error rates in automatic speech recognition and optimizing hearing aid parameters

Martínez, Angel Mario Castro; Gerlach, Lukas; Payá-Vayá, Guillermo; Heřmanský, Hynek; Ooster, Jasper; Meyer, Bernd T.

doi:10.1016/j.specom.2018.11.006

Cited by 17 publications

(8 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, a forward run of the net is relatively cheap and dominated by the multiplications between layers. In a study that analyzed the computational complexity of DNNs and the M-Measure on hearing aid hardware, it was estimated that one forward pass of the DNN and the calculation of the M-Measure can be performed in real time if the number of weights and layers in the DNN are reduced [18]. However, the estimates obtained in [18] are based on a simulation of a typical co-processor, i.e., the corresponding hardware is currently not available.…”

Section: Temporal Integration Time and Computational Requirementsmentioning

confidence: 99%

“…In a related work, the ASQMs that are utilized in the current study were also considered in spatial scenes; however, the main focus was the prediction of word error rates of ASR in spatial scenes and the hardware requirements [18], and DOA estimation was not used in this case. One of the main outcomes was that smaller neural networks for phoneme classification borrowed from an ASR system combined with the M-Measure could be run on hearing aid co-processors in real-time.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Spatial speech detection for binaural hearing aids using deep phoneme classifiers

2022

Self Cite

View full text Add to dashboard Cite

Current hearing aids are limited with respect to speech-specific optimization for spatial sound sources to perform speech enhancement. In this study, we therefore propose an approach for spatial detection of speech based on sound source localization and blind optimization of speech enhancement for binaural hearing aids. We have combined an estimator for the direction of arrival (DOA), featuring high spatial resolution but no specialization to speech, with a measure of speech quality with low spatial resolution obtained after directional filtering. The DOA estimator provides spatial sound source probability in the frontal horizontal plane. The measure of speech quality is based on phoneme representations obtained from a deep neural network, which is part of a hybrid automatic speech recognition (ASR) system. Three ASR-based speech quality measures (ASQM) are explored: entropy, mean temporal distance (M-Measure), matched phoneme (MaP) filtering. We tested the approach in four acoustic scenes with one speaker and either a localized or a diffuse noise source at various signal-to-noise ratios (SNR) in anechoic or reverberant conditions. The effects of incorrect spatial filtering and noise were analyzed. We show that two of the three ASQMs (M-Measure, MaP filtering) are suited to reliably identify the speech target in different conditions. The system is not adapted to the environment and does not require a-priori information about the acoustic scene or a reference signal to estimate the quality of the enhanced speech signal. Nevertheless, our approach performs well in all acoustic scenes tested and varying SNRs and reliably detects incorrect spatial filtering angles.

show abstract

Section: Temporal Integration Time and Computational Requirementsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Spatial speech detection for binaural hearing aids using deep phoneme classifiers

2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…There is a trend towards algorithms, that are computationally more demanding. In recent years, algorithms for machine learning and deep learning [40,44,52] and binaural processing algorithms [46] are used. Recently proposed algorithms of this kind [57,67,69], of which no implementation details on a hearing aid processor are known, are not listed in Table 1.…”

Section: Algorithms For Hearing Aid Devicesmentioning

confidence: 99%

“…Algorithms with a comparably high memory requirements are those based on trained models or data. Among those are localization algorithms [46,60], deep learning based speech enhancement and speech recognitionalgorithms [37,40,44,52]. As an example, the gaussian mixture model (GMM) of the localization algorithm requires about 90% of the total memory requirement of this algorithm [46,60].…”

Section: Work Total Detailsmentioning

confidence: 99%

A Survey on Application Specific Processor Architectures for Digital Hearing Aids

Gerlach

Payá-Vayá

Blume

2021

J Sign Process Syst

Self Cite

View full text Add to dashboard Cite

On the one hand, processors for hearing aids are highly specialized for audio processing, on the other hand they have to meet challenging hardware restrictions. This paper aims to provide an overview of the requirements, architectures, and implementations of these processors. Special attention is given to the increasingly common application-specific instruction-set processors (ASIPs). The main focus of this paper lies on hardware-related aspects such as the processor architecture, the interfaces, the application specific integrated circuit (ASIC) technology, and the operating conditions. The different hearing aid implementations are compared in terms of power consumption, silicon area, and computing performance for the algorithms used. Challenges for the design of future hearing aid processors are discussed based on current trends and developments.

show abstract

“…Performance monitoring (PM) techniques aim for the same goal -to determine the quality of a system's output -based only on the behavior of the system and without any knowledge of the underlying truth. An effective PM measure could be useful in a number of applications [2,3,4,5,6,7,8,9], such as multi-stream selection scenario [4,6,9] or semi-supervised training [2].…”

Section: Introductionmentioning

confidence: 99%

Performance Monitoring for End-to-End Speech Recognition

Sell

Heřmanský

2019

Interspeech 2019

Self Cite

View full text Add to dashboard Cite

Measuring performance of an automatic speech recognition (ASR) system without ground-truth could be beneficial in many scenarios, especially with data from unseen domains, where performance can be highly inconsistent. In conventional ASR systems, several performance monitoring (PM) techniques have been well-developed to monitor performance by looking at triphone posteriors or pre-softmax activations from neural network acoustic modeling. However, strategies for monitoring more recently developed end-to-end ASR systems have not yet been explored, and so that is the focus of this paper. We adapt previous PM measures (Entropy, M-measure and Autoencoder) and apply our proposed RNN predictor in the end-toend setting. These measures utilize the decoder output layer and attention probability vectors, and their predictive power is measured with simple linear models. Our findings suggest that decoder-level features are more feasible and informative than attention-level probabilities for PM measures, and that Mmeasure on the decoder posteriors achieves the best overall predictive performance with an average prediction error 8.8%. Entropy measures and RNN-based prediction also show competitive predictability, especially for unseen conditions.

show abstract

DNN-based performance measures for predicting error rates in automatic speech recognition and optimizing hearing aid parameters

Cited by 17 publications

References 14 publications

Spatial speech detection for binaural hearing aids using deep phoneme classifiers

Spatial speech detection for binaural hearing aids using deep phoneme classifiers

A Survey on Application Specific Processor Architectures for Digital Hearing Aids

Performance Monitoring for End-to-End Speech Recognition

Contact Info

Product

Resources

About