QRD based MVDR beamforming for fast tracking of speech and noise dynamics

Barnov, Anna; Bracha, Vered Bar; Markovich‐Golan, Shmulik

doi:10.1109/waspaa.2017.8170057

Cited by 7 publications

(5 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…4.1. For estimating the high power subspace, we refer to [3] and adopt the procedure for estimating the principal eigenvector, which we present in Sec. 4.2.…”

Section: Recursive Feature Whiteningmentioning

confidence: 99%

See 1 more Smart Citation

Feature Whitening via Gradient Transformation for Improved Convergence

Markovich‐Golan¹,

Battash²,

Bleiweiss³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Feature whitening [16] is a known technique for speeding up training of DNN. Under certain assumptions, whitening the activations [7] reduces the Fisher information matrix to a simple identity matrix, in which case stochastic gradient descent is equivalent to the faster natural gradient descent. Due to the additional complexity resulting from transforming the layer inputs and their corresponding gradients in the forward and backward propagation, and from repeatedly computing the Eigenvalue decomposition (EVD), this method is not commonly used to date. In this work, we address the complexity drawbacks of feature whitening. Our contribution is twofold. First, we derive an equivalent method, which replaces the sample transformations by a transformation to the weight gradients, applied to every batch of B samples. The complexity is reduced by a factor of S/(2B), where S denotes the feature dimension of the layer output. As the batch size increases with distributed training, the benefit of using the proposed method becomes more compelling. Second, motivated by the theoretical relation between the condition number of the sample covariance matrix and the convergence speed, we derive an alternative sub-optimal algorithm which recursively reduces the condition number of the latter matrix. Compared to EVD, complexity is reduced by a factor of the input feature dimension M . We exemplify the proposed algorithms with ResNet-based networks for image classification demonstrated on the CIFAR and Imagenet datasets. Parallelizing the proposed algorithms is straightforward and we implement a distributed version thereof. Improved convergence, in terms of speed and attained accuracy, can be observed in our experiments.Preprint. Under review.

show abstract

“…4.1. For estimating the high power subspace, we refer to [3] and adopt the procedure for estimating the principal eigenvector, which we present in Sec. 4.2.…”

Section: Recursive Feature Whiteningmentioning

confidence: 99%

“…We adopt the procedure from [3] for estimating the principal eigenvector, in the high signal-to-noiseratio case. The block index i is omitted for brevity.…”

Section: High Power Subspace Estimationmentioning

confidence: 99%

Feature Whitening via Gradient Transformation for Improved Convergence

Markovich‐Golan¹,

Battash²,

Bleiweiss³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…These methods were found to handle non-stationary noises effectively [9]. A voice activity detector (VAD) was introduced to estimate the noise during the non-speech, but it failed on encrypted speech signals [10]. Recurrent neural network (RNN) based speech enhancement techniques were introduced.…”

Section: Introductionmentioning

confidence: 99%

A Hybrid Speech Enhancement Algorithm for Voice Assistance Application

Gnanamanickam

Natarajan

Preethaa

2021

Sensors

View full text Add to dashboard Cite

In recent years, speech recognition technology has become a more common notion. Speech quality and intelligibility are critical for the convenience and accuracy of information transmission in speech recognition. The speech processing systems used to converse or store speech are usually designed for an environment without any background noise. However, in a real-world atmosphere, background intervention in the form of background noise and channel noise drastically reduces the performance of speech recognition systems, resulting in imprecise information transfer and exhausting the listener. When communication systems’ input or output signals are affected by noise, speech enhancement techniques try to improve their performance. To ensure the correctness of the text produced from speech, it is necessary to reduce the external noises involved in the speech audio. Reducing the external noise in audio is difficult as the speech can be of single, continuous or spontaneous words. In automatic speech recognition, there are various typical speech enhancement algorithms available that have gained considerable attention. However, these enhancement algorithms work well in simple and continuous audio signals only. Thus, in this study, a hybridized speech recognition algorithm to enhance the speech recognition accuracy is proposed. Non-linear spectral subtraction, a well-known speech enhancement algorithm, is optimized with the Hidden Markov Model and tested with 6660 medical speech transcription audio files and 1440 Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) audio files. The performance of the proposed model is compared with those of various typical speech enhancement algorithms, such as iterative signal enhancement algorithm, subspace-based speech enhancement, and non-linear spectral subtraction. The proposed cascaded hybrid algorithm was found to achieve a minimum word error rate of 9.5% and 7.6% for medical speech and RAVDESS speech, respectively. The cascading of the speech enhancement and speech-to-text conversion architectures results in higher accuracy for enhanced speech recognition. The evaluation results confirm the incorporation of the proposed method with real-time automatic speech recognition medical applications where the complexity of terms involved is high.

show abstract

“…These developments † Work performed while at Reality Labs Research. include approaches that are more computationally efficient [8], probabilistic [9], [10], and able to incorporate both long and short-term information [11]. Several recent works have studied the effects of pose-change in spatially dynamic scenes on conventional beamformers.…”

Section: Introductionmentioning

confidence: 99%

NICE-Beam: Neural Integrated Covariance Estimators for Time-Varying Beamformers

Casebeer¹,

Donley²,

Wong³

et al. 2021

Preprint

View full text Add to dashboard Cite

Estimating a time-varying spatial covariance matrix for a beamforming algorithm is a challenging task, especially for wearable devices, as the algorithm must compensate for time-varying signal statistics due to rapid pose-changes. In this paper, we propose Neural Integrated Covariance Estimators for Beamformers, NICE-Beam. NICE-Beam is a general technique for learning how to estimate time-varying spatial covariance matrices which we apply to joint speech enhancement and dereverberation. It is based on training a neural network module to non-linearly track and leverage scene information across time. We integrate our solution into a beamforming pipeline, which enables simple training, faster than real-time inference, and a variety of test-time adaptation options. We evaluate the proposed model against a suite of baselines in scenes with both stationary and moving microphones. Our results show that the proposed method can outperform a hand-tuned estimator, despite the hand-tuned estimator using oracle source separation knowledge.

show abstract

QRD based MVDR beamforming for fast tracking of speech and noise dynamics

Cited by 7 publications

References 20 publications

Feature Whitening via Gradient Transformation for Improved Convergence

Feature Whitening via Gradient Transformation for Improved Convergence

A Hybrid Speech Enhancement Algorithm for Voice Assistance Application

NICE-Beam: Neural Integrated Covariance Estimators for Time-Varying Beamformers

Contact Info

Product

Resources

About