The Zero Resource Speech Challenge 2019: TTS without T

Dunbar, Ewan; Algayres, Robin; Karadayi, Julien; Bernard, Mathieu; Benjumea, Juan; Cao, Xuan-Nga; Miskic, Lucie; Dugrain, Charlotte; Ondel, Lucas; Black, Alan W.; Besacier, Laurent; Sakti, Sakriani; Dupoux, Emmanuel

doi:10.48550/arxiv.1904.11469

Cited by 14 publications

(24 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Each smaller sequence started and ended with a maximum of 20 ms of non-speech. To make results more comparable to [19], we also train our word segmentation system on the English training set from the ZeroSpeech 2019 Challenge [27] and test on Buckeye. ZeroSpeech English set contains around 15 hours of speech from over 100 speakers.…”

Section: Datasets and Evaluation Metricsmentioning

confidence: 99%

Segmental Contrastive Predictive Coding for Unsupervised Word Segmentation

et al. 2021

View full text Add to dashboard Cite

Automatic detection of phoneme or word-like units is one of the core objectives in zero-resource speech processing. Recent attempts employ self-supervised training methods, such as contrastive predictive coding (CPC), where the next frame is predicted given past context. However, CPC only looks at the audio signal's frame-level structure. We overcome this limitation with a segmental contrastive predictive coding (SCPC) framework that can model the signal structure at a higher level e.g. at the phoneme level. In this framework, a convolutional neural network learns frame-level representation from the raw waveform via noise-contrastive estimation (NCE). A differentiable boundary detector finds variable-length segments, which are then used to optimize a segment encoder via NCE to learn segment representations. The differentiable boundary detector allows us to train frame-level and segment-level encoders jointly. Typically, phoneme and word segmentation are treated as separate tasks. We unify them and experimentally show that our single model outperforms existing phoneme and word segmentation methods on TIMIT and Buckeye datasets. We analyze the impact of boundary threshold and when is the right time to include the segmental loss in the learning process.

show abstract

Section: Datasets and Evaluation Metricsmentioning

confidence: 99%

Segmental Contrastive Predictive Coding for Unsupervised Word Segmentation

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Unit discovery An emerging trend in neural architectures, especially as applied to the speech signal, is the use of mechanisms to enable them to induce discrete, symbol-like internal representations, motivated by concepts such as phonemes and morphemes. Recent editions of the ZeroSpeech challenge (Dunbar et al, 2019) on unit discovery have featured many such approaches. In the visually-grounded setting, (Harwath et al, 2020) adopt the vector-quantization (VQ) approach proposed by van den Oord et al (2017), inserting VQ layers at various points in the speech encoder.…”

Section: Variants and Applicationsmentioning

confidence: 99%

“…One feature of the ABX metric is that it is based on a set of tightly controlled stimuli. Alishahi et al (2017) used synthetic audio; in other work such stimuli have been extracted from utterances using aligned phonemic transcriptions (Dunbar et al, 2019;Anonymous, 2021).…”

Section: Abxmentioning

confidence: 99%

Visually grounded models of spoken language: A survey of datasets, architectures and evaluation techniques

Chrupała

2021

Preprint

View full text Add to dashboard Cite

This survey provides an overview of the evolution of visually grounded models of spoken language over the last 20 years. Such models are inspired by the observation that when children pick up a language, they rely on a wide range of indirect and noisy clues, crucially including signals from the visual modality co-occurring with spoken utterances. Several fields have made important contributions to this approach to modeling or mimicking the process of learning language: Machine Learning, Natural Language and Speech Processing, Computer Vision and Cognitive Science. The current paper brings together these contributions in order to provide a useful introduction and overview for practitioners in all these areas. We discuss the central research questions addressed, the timeline of developments, and the datasets which enabled much of this work. We then summarize the main modeling architectures and offer an exhaustive overview of the evaluation metrics and analysis techniques.

show abstract

“…Each smaller sequence started and ended with a maximum of 20 ms of non-speech. To make results more comparable to [37], we also trained our word segmentation system on the English training set from the ZeroSpeech 2019 Challenge [7] and test on Buckeye. ZeroSpeech English set contains around 15 hours of speech from over 100 speakers.…”

Section: Experiments a Datasets And Evaluation Metricsmentioning

confidence: 99%

“…However, for most spoken languages worldwide, e.g., regional languages, vast amounts of labeled data are not available. Zero Resource speech processing aims to develop alternate techniques that can learn directly from data without any or minimal manual transcriptions [5]- [7]. It This project was supported by NSF Award 1909075 has several applications, including preserving endangered languages, building speech interfaces in low-resource languages, and developing predictive models for understanding language evolution [8].…”

Section: Introductionmentioning

confidence: 99%

Unsupervised Speech Segmentation and Variable Rate Representation Learning using Segmental Contrastive Predictive Coding

Bhati¹,

Villalba²,

Żelasko³

et al. 2021

Preprint

View full text Add to dashboard Cite

Typically, unsupervised segmentation of speech into the phone and word-like units are treated as separate tasks and are often done via different methods which do not fully leverage the inter-dependence of the two tasks. Here, we unify them and propose a technique that can jointly perform both, showing that these two tasks indeed benefit from each other. Recent attempts employ self-supervised learning, such as contrastive predictive coding (CPC), where the next frame is predicted given past context. However, CPC only looks at the audio signal's frame-level structure. We overcome this limitation with a segmental contrastive predictive coding (SCPC) framework to model the signal structure at a higher level, e.g., phone level. A convolutional neural network learns frame-level representation from the raw waveform via noise-contrastive estimation (NCE). A differentiable boundary detector finds variable-length segments, which are then used to optimize a segment encoder via NCE to learn segment representations. The differentiable boundary detector allows us to train frame-level and segment-level encoders jointly. Experiments show that our single model outperforms existing phone and word segmentation methods on TIMIT and Buckeye datasets. We discover that phone class impacts the boundary detection performance, and the boundaries between successive vowels or semivowels are the most difficult to identify. Finally, we use SCPC to extract speech features at the segment level rather than at uniformly spaced frame level (e.g., 10 ms) and produce variable rate representations that change according to the contents of the utterance. We can lower the feature extraction rate from the typical 100 Hz to as low as 14.5 Hz on average while still outperforming the MFCC features on the linear phone classification task.

show abstract

The Zero Resource Speech Challenge 2019: TTS without T

Cited by 14 publications

References 22 publications

Segmental Contrastive Predictive Coding for Unsupervised Word Segmentation

Segmental Contrastive Predictive Coding for Unsupervised Word Segmentation

Visually grounded models of spoken language: A survey of datasets, architectures and evaluation techniques

Unsupervised Speech Segmentation and Variable Rate Representation Learning using Segmental Contrastive Predictive Coding

Contact Info

Product

Resources

About