Overlap-Aware Diarization: Resegmentation Using Neural End-to-End Overlapped Speech Detection

Abstract

We address the problem of effectively handling overlapping speech in a diarization system. First, we detail a neural Long Short-Term Memory-based architecture for overlap detection. Secondly, detected overlap regions are exploited in conjunction with a frame-level speaker posterior matrix to make two-speaker assignments for overlapped frames in the resegmentation step. The overlap detection module achieves state-of-the-art performance on the AMI, DIHARD, and ETAPE corpora. We apply overlap-aware resegmentation on AMI, resulting in a 20% relative DER reduction over the baseline system. While this approach is by no means an endall solution to overlap-aware diarization, it reveals promising directions for handling overlap.

Analysis of the but Diarization System for Voxconverse Challenge

“…We evaluated two approaches for assigning a second speaker. An heuristic that considers the two closest speakers in time [25] and, based on [26], an approach where the second most-likely speaker of the output of VB-HMM diarization is used to provide the second label, but applied using x-vectors as input frames instead of melfrequency cepstral coefficients. Given the current pipeline, obtaining the second label is quite straightforward as we simply need to output the two most likely speakers for each frame.…”

Compositional Embedding Models for Speaker Identification and Diarization with Simultaneous Speech From 2+ Speakers

“…(3) We show that the proposed diarization method achieves state-of-the-art accuracy, slightly outperforming the previous best result [3] on the AMI Headset Mix corpus.…”

“…Multi-person speaker diarization: The most common approach to speaker diarization with simultaneous speech is to use an overlapping speech detector; for those segments that contain overlap, the set of speakers can be estimated [3,9,10,11,12]. For the latter step, one approach is to select the top k closest speakers in the embedding space.…”

“…For example, to test whether x contains speaker a, speaker b, or speakers a&b, we compare f (x) toē a ,ē b ,ē ab and output the set that minimizes the distance. Alternatively, if |T | = k is already known, we can pick the k speakers whose enrollmentsē s are closest to f (x); this is the method used in [3] during overlapping speech.…”

“…[15], one of the largest diarization datasets. In the test set, 81% of the speech is non-overlapping, and 19% is overlapping [3].…”