Study of Senone-Based Deep Neural Network Approaches for Spoken Language Recognition

Ferrer, Luciana; Lei, Yun; McLaren, Mitchell; Scheffer, Nicolas

doi:10.1109/taslp.2015.2496226

Cited by 79 publications

(53 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous work has proven that the statistics of senones can be discriminative in languages [19,10]. We aim for a task-aware version which we name LID-senones, that are constructed from LID-features to make their statistics even more discriminative for LID.…”

Section: Lid-net Structurementioning

confidence: 99%

“…DBFs are inherently robust for different speakers, channels and background noises. Lei et.al, Kenny et.al and Ferrer et.al [8,9,10] proposed collecting sufficient statistics also using structured DNNs to form an effective representation of underlying phonemes or phoneme states. It seems that DNNs are effective for either front-end frame-level feature extraction or back-end utterance-level modelling, where sufficient good quality and quantity training data is available.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

LID-senone Extraction via Deep Neural Networks for End-to-End Language Identification

Ma¹,

Song²,

McLoughlin³

et al. 2016

The Speaker and Language Recognition Workshop (Odyssey 2016)

View full text Add to dashboard Cite

A key problem in spoken language identification (LID) is how to effectively model features from a given speech utterance. Recent techniques such as end-to-end schemes and deep neural networks (DNNs) utilising transfer learning such as bottleneck (BN) features, have demonstrated good overall performance, but have not addressed the extraction of LID-specific features. We thus propose a novel end-to-end neural network which aims to obtain effective LID-senone representations, which we define as being analogous to senones in speech recognition. We show that LID-senones combine a compact representation of the original acoustic feature space with a powerful descriptive and discriminative capability. Furthermore, a novel incremental training method is proposed to extract the weak language information buried in the acoustic features of insufficient language resources. Results on the six most confused languages in NIST LRE 2009 show good performance compared to state-of-the-art BN-GMM/i-vector and BN-DNN/i-vector systems. The proposed end-to-end network, coupled with an incremental training method which mitigates against over-fitting, has potential not just for LID, but also for other resource constrained tasks.

show abstract

Section: Lid-net Structurementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

LID-senone Extraction via Deep Neural Networks for End-to-End Language Identification

Ma¹,

Song²,

McLoughlin³

et al. 2016

The Speaker and Language Recognition Workshop (Odyssey 2016)

View full text Add to dashboard Cite

show abstract

“…Recently, deep encoders have shown effectiveness in factor extraction through a feature learning process. Examples of these features include linguistic features for unsupervised acoustic unit discovery [1], affect-salient features for speech emotion recognition (SER) [2], noise-robust speaker embeddings for speaker recognition (SRE) [3] and phonetically-aware bottleneck features for language recognition (LRE) [4]. This motivates investigations on deep factorization of speech signal [5][6][7][8][9].…”

Section: Introductionmentioning

confidence: 99%

Mixture Factorized Auto-Encoder for Unsupervised Hierarchical Deep Factorization of Speech Signal

Peng

Feng

Lee

2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Speech signal is constituted and contributed by various informative factors, such as linguistic content and speaker characteristic. There have been notable recent studies attempting to factorize speech signal into these individual factors without requiring any annotation. These studies typically assume continuous representation for linguistic content, which is not in accordance with general linguistic knowledge and may make the extraction of speaker information less successful. This paper proposes the mixture factorized auto-encoder (mFAE) for unsupervised deep factorization. The encoder part of mFAE comprises a frame tokenizer and an utterance embedder. The frame tokenizer models linguistic content of input speech with a discrete categorical distribution. It performs frame clustering by assigning each frame a soft mixture label. The utterance embedder generates an utterance-level vector representation. A frame decoder serves to reconstruct speech features from the encoders' outputs. The mFAE is evaluated on speaker verification (SV) task and unsupervised subword modeling (USM) task. The SV experiments on VoxCeleb 1 show that the utterance embedder is capable of extracting speaker-discriminative embeddings with performance comparable to a x-vector baseline. The USM experiments on ZeroSpeech 2017 dataset verify that the frame tokenizer is able to capture linguistic content and the utterance embedder can acquire speaker-related information.Index Termsunsupervised deep factorization, mixture factorized auto-encoder, speaker verification, unsupervised subword modeling

show abstract

“…DBFs are inherently robust to phonotactically irrelevant information. Lei et.al, Kenny et.al and Ferrer et.al [7,8,9] proposed collecting sufficient statistics using a structured DNN to form effective representations from posteriors of phoneme or phoneme states. DNNs have been shown to excel when combined with phonotactic training in LID modelling, nevertheless both the DBFs and calculated statistics are extracted from phoneme or phoneme states, which are not always discriminative to languages.…”

Section: Introductionmentioning

confidence: 99%

End-to-End Language Identification Using High-Order Utterance Representation with Bilinear Pooling

Yan

McLoughlin

et al. 2017

Interspeech 2017

View full text Add to dashboard Cite

A key problem in spoken language identification (LID) is how to design effective representations which are specific to language information. Recent advances in deep neural networks have led to significant improvements in results, with deep endto-end methods proving effective. This paper proposes a novel network which aims to model an effective representation for high (first and second)-order statistics of LID-senones, defined as being LID analogues of senones in speech recognition. The high-order information extracted through bilinear pooling is robust to speakers, channels and background noise. Evaluation with NIST LRE 2009 shows improved performance compared to current state-of-the-art DBF/i-vector systems, achieving over 33% and 20% relative equal error rate (EER) improvement for 3s and 10s utterances and over 40% relative Cavg improvement for all durations.

show abstract

Study of Senone-Based Deep Neural Network Approaches for Spoken Language Recognition

Cited by 79 publications

References 29 publications

LID-senone Extraction via Deep Neural Networks for End-to-End Language Identification

LID-senone Extraction via Deep Neural Networks for End-to-End Language Identification

Mixture Factorized Auto-Encoder for Unsupervised Hierarchical Deep Factorization of Speech Signal

End-to-End Language Identification Using High-Order Utterance Representation with Bilinear Pooling

Contact Info

Product

Resources

About