Variational Domain Adversarial Learning With Mutual Information Maximization for Speaker Verification

Tu, Youzhi; Mak, Man‐Wai; Chien, Jen‐Tzung

doi:10.1109/taslp.2020.3004760

Cited by 28 publications

(11 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To fully characterize dual translation between two domains, only considering domain mapping is not enough. Representing rich information within individual domains and between two domains is equally important [24]. It is crucial to represent domain mapping as well as domain knowledge and also integrate them in a unified transformer for seq2seq learning.…”

Section: Domain Mapping Knowledge and Dual Integrationmentioning

confidence: 99%

Dualformer: A Unified Bidirectional Sequence-to-Sequence Learning

Chien

Chang

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

This paper presents a new dual domain mapping based on a unified bidirectional sequence-to-sequence (seq2seq) learning. Traditionally, dual learning in domain mapping was constructed with intrinsic connection where the conditional generative models in two directions were mutually leveraged and combined. The additional feedback from the other generation direction was used to regularize sequential learning in original direction of domain mapping. Domain matching between source sequence and target sequence was accordingly improved. However, the reconstructions for knowledge in two domains were ignored. The dual information based on separate models in two training directions was not sufficiently discovered. To cope with this weakness, this study proposes a closed-loop seq2seq learning where domain mapping and domain knowledge are jointly learned. In particular, a new feature-level dual learning is incorporated to build a dualformer where feature integration and feature reconstruction are further performed to bridge dual tasks. Experiments demonstrate the merit of the proposed dualformer for machine translation based on the multi-objective seq2seq learning.

show abstract

Section: Domain Mapping Knowledge and Dual Integrationmentioning

confidence: 99%

Dualformer: A Unified Bidirectional Sequence-to-Sequence Learning

Chien

Chang

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…The new learning machines, e.g. Bayesian recurrent network [10,19], VAE [27], neural variational learning [14,15,31], neural discrete representation [24,34], stochastic layered model [18,21,33], stochastic temporal network [1], temporal difference VAE [23], Markov recurrent network [28], temporal-difference network, and neural ordinary differential equation [3] are introduced in various deep models which pave an avenue to knowledge representation and information discovery. Variational inference methods based on normalizing flows [26,30] and variational mixture of posteriors [32] are addressed.…”

Section: Bayesian Information Processingmentioning

confidence: 99%

Neural Bayesian Information Processing

Chien

2020

Proceedings of the 29th ACM International Conference on Information &Amp; Knowledge Management

Self Cite

View full text Add to dashboard Cite

Deep learning is developed as a learning process from source inputs to target outputs where the inference or optimization is performed over an assumed deterministic model with deep structure. A wide range of temporal and spatial data in language and vision are treated as the inputs or outputs to build such a complicated mapping in different information systems. A systematic and elaborate transfer is required to meet the mapping between source and target domains. Also, the semantic structure in natural language and computer vision may not be well represented or trained in mathematical logic or computer programs. The distribution function in discrete or continuous latent variable model for words, sentences, images or videos may not be properly decomposed or estimated. The system robustness to heterogeneous environments may not be assured. This tutorial addresses the fundamentals and advances in statistical models and neural networks, and presents a series of deep Bayesian solutions including variational Bayes, sampling method, Bayesian neural network, variational auto-encoder (VAE), stochastic recurrent neural network, sequence-to-sequence model, attention mechanism, end-to-end network, stochastic temporal convolutional network, temporal difference VAE, normalizing flow and neural ordinary differential equation. Enhancing the prior/posterior representation is addressed in different latent variable models. We illustrate how these models are connected and why they work for a variety of applications on complex patterns in language and vision. The word, sentence and image embeddings are merged with semantic constraint or structural information. Bayesian learning is formulated in the optimization procedure where the posterior collapse is tackled. An informative latent space is trained to incorporate deep Bayesian learning in various information systems.

show abstract

“…It has been shown that the xvectors are more speaker discriminative than the i-vectors. They are also more robust to noise, reverberation, and domain mismatch [6][7][8][9]. There are also embedding systems that use ResNets [10] or DenseNets [11] for frame-level processing.…”

Section: Introductionmentioning

confidence: 99%

Short-Time Spectral Aggregation for Speaker Embedding

Mak

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

State-of-the-art speaker verification systems take frame-level acoustics features as input and produce fixed-dimensional embeddings as utterance-level representations. Thus, how to aggregate information from frame-level features is vital for achieving high performance. This paper introduces short-time spectral pooling (STSP) for better aggregation of frame-level information. STSP transforms the temporal feature maps of a speaker embedding network into the spectral domain and extracts the lowest spectral components of the averaged spectrograms for aggregation. Benefiting from the low-pass characteristic of the averaged spectrograms, STSP is able to preserve most of the speaker information in the feature maps using a few spectral components only. We show that statistics pooling is a special case of STSP where only the DC spectral components are used. Experiments on VoxCeleb1 and VOiCES 2019 show that STSP outperforms statistics pooling and multi-head attentive pooling, which suggests that leveraging more spectral information in the CNN feature maps can produce highly discriminative speaker embeddings.

show abstract

Variational Domain Adversarial Learning With Mutual Information Maximization for Speaker Verification

Cited by 28 publications

References 30 publications

Dualformer: A Unified Bidirectional Sequence-to-Sequence Learning

Dualformer: A Unified Bidirectional Sequence-to-Sequence Learning

Neural Bayesian Information Processing

Short-Time Spectral Aggregation for Speaker Embedding

Contact Info

Product

Resources

About