CountNet: Estimating the Number of Concurrent Speakers Using Supervised Learning

Abstract

Estimating the maximum number of concurrent speakers from single-channel mixtures is a challenging problem and an essential first step to address various audio-based tasks such as blind source separation, speaker diarization, and audio surveillance. We propose a unifying probabilistic paradigm, where deep neural network architectures are used to infer output posterior distributions. These probabilities are in turn processed to yield discrete point estimates. Designing such architectures often involves two important and complementary aspects that we investigate and discuss. First, we study how recent advances in deep architectures may be exploited for the task of speaker count estimation. In particular, we show that convolutional recurrent neural networks outperform recurrent networks used in a previous study when adequate input features are used. Even for short segments of speech mixtures, we can estimate up to five speakers, with a significantly lower error than other methods. Second, through comprehensive evaluation, we compare the best-performing method to several baselines, as well as the influence of gain variations, different datasets, and reverberation. The output of our proposed method is compared to human performance. Finally, we give insights into the strategy used by our proposed method.

Application of Machine Learning in Microbiology

“…ML methods can be divided into two types (Zitnik et al, 2019), supervised learning and unsupervised learning. Supervised learning (Stoter et al, 2019) requires that the model be trained using a training set. The training sets for supervised learning include features and results.…”

Detecting and Counting Overlapping Speakers in Distant Speech Scenarios

“…Speaker counting can be formulated as an (N + 1)-classes classification problem with N the maximum possible number of overlapping speakers [21]. While this approach is not the only one for supervised speaker counting, it has been found to be the most effective [22], provided the maximum possible number is known.…”

“…In parallel, Stöter et al [21] have shown that a neural network can be trained to estimate the number of concurrent speakers rather than simply performing joint VAD+OSD. This approach has been further expanded in [22] where three different output distributions for this speaker counting problem are proposed, different neural architectures are explored, and the performance is compared with humans. Also, in [23], a deep learning based speaker counting algorithm was evaluated against…”

“…Building on these previous works, we study supervised joint VAD+OSD and speaker counting in distant speech scenarios. We propose a Temporal Convolutional Network (TCN) architecture for these tasks, and evaluate it against previous works on joint VAD+OSD [17] and speaker counting [22] on AMI and CHiME-6 [14]. Because, to the best of our knowledge, supervised speaker counting has never been studied on real-world data, we also explore how the class imbalance problem can be mitigated using data augmentation.…”

Estimating Number of Speakers via Density-Based Clustering and Classification Decision

“…In addition, various methods based on deep learning on counting the NoS have emerged, such as [18]- [21]. In [22], a new NoS estimation architecture is provided via combining the convolutional recurrent neural networks and adequate input features of speeches, which is designed to improve the performance of NoS estimation from the single channel mixtures.…”