A K-Nearest Neighbours Approach To Unsupervised Spoken Term Discovery

Abstract

Unsupervised spoken term discovery is the task of finding recurrent acoustic patterns in speech without any annotations. Current approaches consists of two steps: (1) discovering similar patterns in speech, and (2) partitioning those pairs of acoustic tokens using graph clustering methods. We propose a new approach for the first step. Previous systems used various approximation algorithms to make the search tractable on large amounts of data. Our approach is based on an optimized k-nearest neighbours (KNN) search coupled with a fixed word embedding algorithm. The results show that the KNN algorithm is robust across languages, consistently outperforms the DTW-based baseline, and is competitive with current state-of-the-art spoken term discovery systems.

Evaluating the Reliability of Acoustic Speech Embeddings

“…Finding positive pairs of speech sequences is an area of research called unsupervised term discovery (UTD) [16][17][18]28]. Such UTD systems can be DTW alignment based [16] or involve a k-Nearest-Neighbours search [28]. We opted for the latter, as it is both scalable and among the state-of-the-art methods.…”

“…We opted for the latter, as it is both scalable and among the state-of-the-art methods. It encodes exhaustively all possible speech sequences with an embedding model, and used optimised k-NN search [29] to retrieve acoustically similar pairs of speech sequences (see the details in [28]). In our experiments, we used the pairs retrieved by k-NN on GD-PLP encoded sequences to train our self-supervised models (CAE,Siamese, CAE-Siamese).…”

“…Each corpus was split into all possible segmentations to produce random speech sequences as described in [28]. Random speech sequences span from 70ms to 1s.…”

“…The speaker embedding network is a single fully connected layer with fifteen neurons. Our UTD system [28] uses the embeddings of the GD-PLP model. A set of speech pairs is returned, sorted by cosine similarity.…”

Speech Sequence Embeddings using Nearest Neighbors Contrastive Learning

“…The model is a convolution and transformer-based embedder trained with the NTXEnt contrastive loss [22]. Building on similar ideas in vision and speech, we select our positive examples through a mix of time-stretching data augmentation [23] and k-Nearerst Neighbors search [24,25]. Figure 1 gives an overview of our method.…”