Bayesian Subspace Hidden Markov Model for Acoustic Unit Discovery

Ondel, Lucas; Vydana, Hari Krishna; Burget, Lukáš; Černocký, Jaň

doi:10.48550/arxiv.1904.03876

Cited by 7 publications

(5 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A model that can do this can also be used for unsupervised phone segmentation, predicting phone boundaries. Several acoustic unit discovery and unsupervised phone segmentation models have been proposed [22]- [27]. Recently, large gains have been achieved by combining selfsupervised neural networks with a clustering component.…”

Section: Dpdp For Acoustic Unit Discoverymentioning

confidence: 99%

Word Segmentation on Discovered Phone Units with Dynamic Programming and Self-Supervised Scoring

Kamper¹

2022

Preprint

View full text Add to dashboard Cite

Recent work on unsupervised speech segmentation has used self-supervised models with a phone segmentation module and a word segmentation module that are trained jointly. This paper compares this joint methodology with an older idea: bottom-up phone-like unit discovery is performed first, and symbolic word segmentation is then performed on top of the discovered units (without influencing the lower level). I specifically describe a duration-penalized dynamic programming (DPDP) procedure that can be used for either phone or word segmentation by changing the self-supervised scoring network that gives segment costs. For phone discovery, DPDP is applied with a contrastive predictive coding clustering model, while for word segmentation it is used with an autoencoding recurrent neural network. The two models are chained in order to segment speech. This approach gives comparable word segmentation results to state-of-the-art joint self-supervised models on an English benchmark. On French and Mandarin data, it outperforms previous systems on the ZeroSpeech benchmarks. Analysis shows that the chained DPDP system segments shorter filler words well, but longer words might require an external top-down signal.

show abstract

Section: Dpdp For Acoustic Unit Discoverymentioning

confidence: 99%

Word Segmentation on Discovered Phone Units with Dynamic Programming and Self-Supervised Scoring

Kamper¹

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Most importantly, our work is inspired by studies showing the benefit of using multilingual bottleneck features as framelevel representations for zero-resource languages [38]- [41]: a frame-level network is trained jointly on several well-resourced languages (normally to predict context-dependent triphone HMM states) and is then applied to an unseen language. In [42], multilingual data was also used for discovering acoustic units. As in these studies, our findings show the advantage of learning from labelled data in well-resourced languages when processing an unseen low-resource language-here at the word rather than subword level.…”

Section: Related Workmentioning

confidence: 99%

Improved Acoustic Word Embeddings for Zero-Resource Languages Using Multilingual Transfer

Kamper

Matusevych

Goldwater

2021

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

Acoustic word embeddings are fixed-dimensional representations of variable-length speech segments. Such embeddings can form the basis for speech search, indexing and discovery systems when conventional speech recognition is not possible. In zero-resource settings where unlabelled speech is the only available resource, we need a method that gives robust embeddings on an arbitrary language. Here we explore multilingual transfer: we train a single supervised embedding model on labelled data from multiple well-resourced languages and then apply it to unseen zero-resource languages. We consider three multilingual recurrent neural network (RNN) models: a classifier trained on the joint vocabularies of all training languages; a Siamese RNN trained to discriminate between same and different words from multiple languages; and a correspondence autoencoder (CAE) RNN trained to reconstruct word pairs. In a word discrimination task on six target languages, all of these models outperform state-of-the-art unsupervised models trained on the zero-resource languages themselves, giving relative improvements of more than 30% in average precision. When using only a few training languages, the multilingual CAE performs better, but with more training languages the other multilingual models perform similarly. Using more training languages is generally beneficial, but improvements are marginal on some languages. We present probing experiments which show that the CAE encodes more phonetic, word duration, language identity and speaker information than the other multilingual models.

show abstract

“…A zero-resource language is one for which no transcribed speech resources are available for developing speech systems [1,2]. Although conventional speech recognition is not possible for such languages, researchers have shown how speech search [3][4][5], discovery [6][7][8][9], and segmentation and clustering [10][11][12] applications can be developed without any labelled speech audio. In many of these applications, a metric is required for comparing speech segments of different durations.…”

Section: Introductionmentioning

confidence: 99%

Acoustic Word Embeddings for Zero-Resource Languages Using Self-Supervised Contrastive Learning and Multilingual Adaptation

Jacobs

Matusevych

Kamper

2021

2021 IEEE Spoken Language Technology Workshop (SLT)

View full text Add to dashboard Cite

Acoustic word embeddings (AWEs) are fixed-dimensional representations of variable-length speech segments. For zeroresource languages where labelled data is not available, one AWE approach is to use unsupervised autoencoder-based recurrent models. Another recent approach is to use multilingual transfer: a supervised AWE model is trained on several well-resourced languages and then applied to an unseen zero-resource language. We consider how a recent contrastive learning loss can be used in both the purely unsupervised and multilingual transfer settings. Firstly, we show that terms from an unsupervised term discovery system can be used for contrastive self-supervision, resulting in improvements over previous unsupervised monolingual AWE models. Secondly, we consider how multilingual AWE models can be adapted to a specific zero-resource language using discovered terms. We find that self-supervised contrastive adaptation outperforms adapted multilingual correspondence autoencoder and Siamese AWE models, giving the best overall results in a word discrimination task on six zero-resource languages.

show abstract

Bayesian Subspace Hidden Markov Model for Acoustic Unit Discovery

Cited by 7 publications

References 0 publications

Word Segmentation on Discovered Phone Units with Dynamic Programming and Self-Supervised Scoring

Word Segmentation on Discovered Phone Units with Dynamic Programming and Self-Supervised Scoring

Improved Acoustic Word Embeddings for Zero-Resource Languages Using Multilingual Transfer

Acoustic Word Embeddings for Zero-Resource Languages Using Self-Supervised Contrastive Learning and Multilingual Adaptation

Contact Info

Product

Resources

About