The 2015 sheffield system for longitudinal diarisation of broadcast media

Abstract

Speaker diarisation is the task of answering "who spoke when" within a multi-speaker audio recording. Diarisation of broadcast media typically operates on individual television shows, and is a particularly difficult task, due to a high number of speakers and challenging background conditions. Using prior knowledge, such as that from previous shows in a series, can improve performance. Longitudinal diarisation allows to use knowledge from previous audio files to improve performance, but requires finding matching speakers across consecutive files. This paper describes the University of Sheffield system for participation in the 2015 Multi-Genre Broadcast (MGB) challenge. The challenge required longitudinal diarisation of data from BBC archives, under very constrained resource settings. Our system consists of three main stages: speech activity detection using DNNs with novel adaptation and decoding methods; speaker segmentation and clustering, with adaptation of the DNNbased clustering models; and finally speaker linking to match speakers across shows. The final result on the development set of 19 shows from five different television series provided a Diarisation Error Rate of 50.77% in the diarisation and linking task.

The MGB challenge: Evaluating multi-genre broadcast media recognition

“…• SU [30]: clustering using the SHoUT toolkit; posteriors are generated from a Speaker Separation DNN, used as input to a speaker-state HMM.…”

DNN-Based Speaker Clustering for Speaker Diarisation

“…DNNs which can classify speakers have been implemented in the speaker recognition field [11] which involves projecting acoustic features into a lower-dimensional feature set. Our own previous work has also implemented speaker separation DNNs [12].…”

DNN approach to speaker diarisation using speaker channels

“…A speaker segmentation stage using autoassociative neural networks (AANN) was proposed in which a windowing method is used where an AANN model is trained for the left half of the window and tested on the right to give a confidence score on how likely each part belongs to the same speaker [17]. Finally, DNNs have been applied to the clustering stage by training speaker separation DNNs and adapting these to specific recordings [16,18].…”

“…Artificial neural networks (ANN) have been trained to learn a feature transform [15] and DNNs can be trained to detect speech/nonspeech in an SAD stage where adapting the DNN leads to improved performance [16]. A speaker segmentation stage using autoassociative neural networks (AANN) was proposed in which a windowing method is used where an AANN model is trained for the left half of the window and tested on the right to give a confidence score on how likely each part belongs to the same speaker [17].…”