Feature Enhancement with Deep Feature Losses for Speaker Verification

Kataria, Saurabh; Nidadavolu, Phani Sankar; Villalba, Jesús; Chen, Nanxin; Garcia-Perera, Paola; Dehak, Najim

doi:10.1109/icassp40776.2020.9053110

Cited by 22 publications

(25 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They are used as the high level abstraction to measure the training loss between reconstructed signals and reference signals. Such training loss is also called deep feature loss [71], [72].…”

Section: Perceptual Loss For Style Reconstructionmentioning

confidence: 99%

Expressive TTS Training With Frame and Style Reconstruction Loss

Liu

Şişman

Gao

et al. 2021

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

We propose a novel training strategy for Tacotronbased text-to-speech (TTS) system that improves the speech styling at utterance level. One of the key challenges in prosody modeling is the lack of reference that makes explicit modeling difficult. The proposed technique doesn't require prosody annotations from training data. It doesn't attempt to model prosody explicitly either, but rather encodes the association between input text and its prosody styles using a Tacotron-based TTS framework. This study marks a departure from the style token paradigm where prosody is explicitly modeled by a bank of prosody embeddings. It adopts a combination of two objective functions: 1) frame level reconstruction loss, that is calculated between the synthesized and target spectral features; 2) utterance level style reconstruction loss, that is calculated between the deep style features of synthesized and target speech. The style reconstruction loss is formulated as a perceptual loss to ensure that utterance level speech style is taken into consideration during training. Experiments show that the proposed training strategy achieves remarkable performance and outperforms the state-ofthe-art baseline in both naturalness and expressiveness. To our best knowledge, this is the first study to incorporate utterance level perceptual quality as a loss function into Tacotron training for improved expressiveness.

show abstract

Section: Perceptual Loss For Style Reconstructionmentioning

confidence: 99%

Expressive TTS Training With Frame and Style Reconstruction Loss

Liu

Şişman

Gao

et al. 2021

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

show abstract

“…Nevertheless, these methods are not task-specific, and the training data of speech enhancement and speaker embedding might belong to different domains. In [14], researchers evaluate and optimize the speech enhancement model based on perceptual loss, which is calculated by a pre-trained speaker embedding network. In [15,16,17], researchers connect and train the speech enhancement and speaker embedding networks in an end-to-end manner.…”

Section: Introductionmentioning

confidence: 99%

Cam: Context-Aware Masking for Robust Speaker Verification

Zheng

Suo

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Performance degradation caused by noise has been a long-standing challenge for speaker verification. Previous methods usually involve applying a denoising transformation to speaker embeddings or enhancing input features. Nevertheless, these methods are lossy and inefficient for speaker embedding. In this paper, we propose contextaware masking (CAM), a novel method to extract robust speaker embedding. CAM enables the speaker embedding network to "focus" on the speaker of interest and "blur" unrelated noise. The threshold of masking is dynamically controlled by an auxiliary context embedding that captures speaker and noise characteristics. Moreover, models adopting CAM can be trained in an end-to-end manner without using synthesized noisy-clean speech pairs. Our results show that CAM improves speaker verification performance in the wild by a large margin, compared to the baselines.

show abstract

“…As the input of the ResNet network, three-channel (RGB) images were used, so the spectrograms were converted to RGB images. Another study in which MFCCs were used as the input of a DNN was developed to construct x-vector embeddings for speaker verification tasks [ 40 ]. In addition, in several works, the TDNNs were exploited to recognize speech [ 41 ], emotion [ 42 ], speaker, or voice activity.…”

Section: Introductionmentioning

confidence: 99%

An Incremental Class-Learning Approach with Acoustic Novelty Detection for Acoustic Event Recognition

Bayram

İnce

2021

Sensors

View full text Add to dashboard Cite

Acoustic scene analysis (ASA) relies on the dynamic sensing and understanding of stationary and non-stationary sounds from various events, background noises and human actions with objects. However, the spatio-temporal nature of the sound signals may not be stationary, and novel events may exist that eventually deteriorate the performance of the analysis. In this study, a self-learning-based ASA for acoustic event recognition (AER) is presented to detect and incrementally learn novel acoustic events by tackling catastrophic forgetting. The proposed ASA framework comprises six elements: (1) raw acoustic signal pre-processing, (2) low-level and deep audio feature extraction, (3) acoustic novelty detection (AND), (4) acoustic signal augmentations, (5) incremental class-learning (ICL) (of the audio features of the novel events) and (6) AER. The self-learning on different types of audio features extracted from the acoustic signals of various events occurs without human supervision. For the extraction of deep audio representations, in addition to visual geometry group (VGG) and residual neural network (ResNet), time-delay neural network (TDNN) and TDNN based long short-term memory (TDNN–LSTM) networks are pre-trained using a large-scale audio dataset, Google AudioSet. The performances of ICL with AND using Mel-spectrograms, and deep features with TDNNs, VGG, and ResNet from the Mel-spectrograms are validated on benchmark audio datasets such as ESC-10, ESC-50, UrbanSound8K (US8K), and an audio dataset collected by the authors in a real domestic environment.

show abstract

Feature Enhancement with Deep Feature Losses for Speaker Verification

Cited by 22 publications

References 23 publications

Expressive TTS Training With Frame and Style Reconstruction Loss

Expressive TTS Training With Frame and Style Reconstruction Loss

Cam: Context-Aware Masking for Robust Speaker Verification

An Incremental Class-Learning Approach with Acoustic Novelty Detection for Acoustic Event Recognition

Contact Info

Product

Resources

About