Speaker Diarization: A Review of Recent Research

Abstract

Abstract-Speaker diarization is the task of determining "who spoke when?" in an audio or video recording that contains an unknown amount of speech and also an unknown number of speakers. Initially, it was proposed as a research topic related to automatic speech recognition, where speaker diarization serves as an upstream processing step. Over recent years, however, speaker diarization has become an important key technology for many tasks, such as navigation, retrieval, or higher-level inference on audio data. Accordingly, many important improvements in accuracy and robustness have been reported in journals and conferences in the area. The application domains, from broadcast news, to lectures and meetings, vary greatly and pose different problems, such as having access to multiple microphones and multimodal information or overlapping speech. The most recent review of existing technology dates back to 2006 and focuses on the broadcast news domain. In this paper we review the current state-of-the-art, focusing on research developed since 2006 that relates predominantly to speaker diarization for conference meetings. Finally, we present an analysis of speaker diarization performance as reported through the NIST Rich Transcription evaluations on meeting data and identify important areas for future research.

Where are the challenges in speaker diarization?

“…This error is referred to as speech time error in the results computed by NIST tools 1 . Others [13] choose to report the FA speaker and Miss speaker errors inclusive of overlap, e.g. a segment which contains two speakers that has been completely missed by the system will have twice the error.…”

DNN approach to speaker diarisation using speaker channels

“…The task of speaker diarisation is an important prerequisite task for audio indexing, automatic speech recognition (ASR) and more [1,2]. The objective is to split the audio into segments which are associated with a single speaker, and to identify among the set of segments those that are spoken by the same speaker.…”

Multimodal corpus of multiparty conversations in L1 and L2 languages and findings obtained from it

“…Speaker diarization (the process of partitioning an input audio stream into homogeneous segments according to speaker identity), when used together with speaker recognition (the identification of speakers by their voices), has become an important key technology for tasks such as navigation, retrieval, and high-level inference from audio data in meeting recordings. Some speaker diarization systems integrate motion and gazing data analyses with audio data analysis to achieve higher accuracy and robustness (Anguera et al 2012;Moattar and Homayounpour 2012). There are also meeting systems that use multimodal data including both motion and gaze (Hain et al 2010;Tur et al 2008).…”