Segmental Audio Word2Vec: Representing Utterances as Sequences of Vectors with Applications in Spoken Term Detection

Wang, Yu-Hsuan; Lee, Hung-yi; Lee, Lin-Shan

doi:10.1109/icassp.2018.8462002

Cited by 50 publications

(46 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Term discovery systems were also used to provide training segments for a speech model in [22]. Although their systems are unsupervised, the intrinsic quality of embeddings was not their direct focus, as is also the case in [11,20]. Another study where ground truth segments are not used is [24], although true phoneme boundaries are assumed.…”

Section: Encoder-decoder Correspondence Autoencodermentioning

confidence: 99%

Truly Unsupervised Acoustic Word Embeddings Using Weak Top-down Constraints in Encoder-decoder Models

Kamper

2019

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

106

View full text Add to dashboard Cite

We investigate unsupervised models that can map a variableduration speech segment to a fixed-dimensional representation. In settings where unlabelled speech is the only available resource, such acoustic word embeddings can form the basis for "zero-resource" speech search, discovery and indexing systems. Most existing unsupervised embedding methods still use some supervision, such as word or phoneme boundaries. Here we propose the encoder-decoder correspondence autoencoder (ENCDEC-CAE), which, instead of true word segments, uses automatically discovered segments: an unsupervised term discovery system finds pairs of words of the same unknown type, and the ENCDEC-CAE is trained to reconstruct one word given the other as input. We compare it to a standard encoderdecoder autoencoder (AE), a variational AE with a prior over its latent embedding, and downsampling. ENCDEC-CAE outperforms its closest competitor by 29% relative in average precision on two languages in a word discrimination task.Index Terms-Acoustic word embeddings, zero-resource speech processing, unsupervised learning, query-by-example.

show abstract

Section: Encoder-decoder Correspondence Autoencodermentioning

confidence: 99%

Truly Unsupervised Acoustic Word Embeddings Using Weak Top-down Constraints in Encoder-decoder Models

Kamper

2019

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

106

View full text Add to dashboard Cite

show abstract

“…Our approach to semantic QbE is embedding-based: We learn an embedding function that maps from segments of speech-queries, search utterances, or sub-segments of search utterances-to fixed-dimensional vectors; we search for semantic matches by finding the minimum distance between query and search utterance embedding vectors. In this respect our approach is similar to those in recent embedding-based QbE work [14][15][16][17], and also some embedding-based spoken term detection work [18]. The key difference is that our embedding function must be learned in such a way that similar embedding vectors are semantically rather than phonetically similar.…”

Section: Introductionmentioning

confidence: 87%

Semantic Query-by-example Speech Search Using Visual Grounding

Kamper

Anastassiou

Livescu

2019

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

A number of recent studies have started to investigate how speech systems can be trained on untranscribed speech by leveraging accompanying images at training time. Examples of tasks include keyword prediction and within-and acrossmode retrieval. Here we consider how such models can be used for query-by-example (QbE) search, the task of retrieving utterances relevant to a given spoken query. We are particularly interested in semantic QbE, where the task is not only to retrieve utterances containing exact instances of the query, but also utterances whose meaning is relevant to the query. We follow a segmental QbE approach where variable-duration speech segments (queries, search utterances) are mapped to fixeddimensional embedding vectors. We show that a QbE system using an embedding function trained on visually grounded speech data outperforms a purely acoustic QbE system in terms of both exact and semantic retrieval performance.

show abstract

“…There are many approaches for segmenting utterances automatically. Automatic segmentation of spoken words has been successfully trained and reported previously [32], so the training audio corpus in the present work has been previously segmented into phonetic words. A word and its corresponding phonetics form a token.…”

Section: A Stage 1: Phonetic Representation Acquisitionmentioning

confidence: 99%

A Synchronized Word Representation Method With Dual Perceptual Information

Zhu

et al. 2020

IEEE Access

View full text Add to dashboard Cite

The information used for human natural language comprehension is usually perceptual information, such as text, sounds, and images. In recent years, language models that learn semantics from single perceptual information sources (text) have gradually developed into multimodal language models that learn semantics from multiple perceptual information sources. Sound is perceptual information other than text that has been proven effective by many related works. However, there is still a need for further research on the incorporation method for perceptual information. Thus, this paper proposes a language model that synchronously trains dual perceptual information to enhance word representation. The representation is trained in a synchronized way that adopts an attention model to utilize both text and phonetic perceptual information in unsupervised learning tasks. On basis of that, these dual perceptual information is processed simultaneously, and that is similar with the cognitive process of human language understanding. The experiment results show that our approach achieve superior results in text classification and word similarity tasks with four languages of data set. INDEX TERMS Information representation, multi-layer neural network, natural language processing, unsupervised learning.

show abstract

Segmental Audio Word2Vec: Representing Utterances as Sequences of Vectors with Applications in Spoken Term Detection

Cited by 50 publications

References 13 publications

Truly Unsupervised Acoustic Word Embeddings Using Weak Top-down Constraints in Encoder-decoder Models

Truly Unsupervised Acoustic Word Embeddings Using Weak Top-down Constraints in Encoder-decoder Models

Semantic Query-by-example Speech Search Using Visual Grounding

A Synchronized Word Representation Method With Dual Perceptual Information

Contact Info

Product

Resources

About