BUT System for DIHARD Speech Diarization Challenge 2018

Díez, Mireia; Landini, Federico; Burget, Lukáš; Rohdin, Johan; Silnova, Anna; Žmolíková, Kateřina; Novotny, Ondrej; Veselý, Karel; Glembek, Ondřej; Plchot, Oldřich; Mošner, Ladislav; Matějka, Pavel

doi:10.21437/interspeech.2018-1749

Cited by 46 publications

(44 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The speech enhancement module is used only for tracks 2 and 4 as a pre-processing front-end for the SAD pipeline as the diarization system did not show improvements using the enhanced audio. The scores obtained by the challenge baseline are quite high, with track 1 DER roughly in line with the performance of the best DIHARD I systems [14,15,25] and track 2 DER 5% higher than for DIHARD I (15% without enhancement), which we suspect reflects a combination of superior SAD components in those systems and the more careful segmentation for the child language and web video domains in DIHARD II. Error rates are noticeably higher for tracks 3 and 4, reaching 50.85% and 77.34% respectively, though, again, these rates are roughly in line with those observed for the best DIHARD I systems on the two most difficult domains in that challenge: restaurant and child language.…”

Section: Baseline Resultssupporting

confidence: 61%

The Second DIHARD Diarization Challenge: Dataset, Task, and Baselines

Ryant¹,

Church²,

Cieri³

et al. 2019

Interspeech 2019

140

117

View full text Add to dashboard Cite

This paper introduces the second DIHARD challenge, the second in a series of speaker diarization challenges intended to improve the robustness of diarization systems to variation in recording equipment, noise conditions, and conversational domain. The challenge comprises four tracks evaluating diarization performance under two input conditions (single channel vs. multi-channel) and two segmentation conditions (diarization from a reference speech segmentation vs. diarization from scratch). In order to prevent participants from overtuning to a particular combination of recording conditions and conversational domain, recordings are drawn from a variety of sources ranging from read audiobooks to meeting speech, to child language acquisition recordings, to dinner parties, to web video. We describe the task and metrics, challenge design, datasets, and baseline systems for speech enhancement, speech activity detection, and diarization. 1 See, for instance, the release of IBM's diarization API in 2017. The feature worked well for simple cases, but when run by users on real inputs, the performance was found to be lacking, especially for overlaps, back-channels, and short turns.

show abstract

Section: Baseline Resultssupporting

confidence: 61%

The Second DIHARD Diarization Challenge: Dataset, Task, and Baselines

Ryant¹,

Church²,

Cieri³

et al. 2019

Interspeech 2019

140

117

View full text Add to dashboard Cite

show abstract

“…These clustering-based diarization methods have shown themselves to be The first author performed the work while at Center for Language and Speech Processing, Johns Hopkins University as a Visiting Scholar. effective on various datasets (see the DIHARD Challenge 2018 activities, e.g., [23][24][25]).…”

Section: Introductionmentioning

confidence: 99%

End-to-End Neural Speaker Diarization with Self-Attention

Fujita

Kanda

Horiguchi

et al. 2019

2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)

164

175

View full text Add to dashboard Cite

Speaker diarization has been mainly developed based on the clustering of speaker embeddings. However, the clustering-based approach has two major problems; i.e., (i) it is not optimized to minimize diarization errors directly, and (ii) it cannot handle speaker overlaps correctly. To solve these problems, the End-to-End Neural Diarization (EEND), in which a bidirectional long short-term memory (BLSTM) network directly outputs speaker diarization results given a multi-talker recording, was recently proposed. In this study, we enhance EEND by introducing self-attention blocks instead of BLSTM blocks. In contrast to BLSTM, which is conditioned only on its previous and next hidden states, self-attention is directly conditioned on all the other frames, making it much suitable for dealing with the speaker diarization problem. We evaluated our proposed method on simulated mixtures, real telephone calls, and real dialogue recordings. The experimental results revealed that the self-attention was the key to achieving good performance and that our proposed method performed significantly better than the conventional BLSTM-based method. Our method was even better than that of the state-of-the-art x-vector clustering-based method. Finally, by visualizing the latent representation, we show that the self-attention can capture global speaker characteristics in addition to local speech activity dynamics. Our source code is available online at https://github.com/hitachi-speech/EEND. Index Termsspeaker diarization, neural network, end-to-end, self-attention arXiv:1909.06247v1 [eess.AS] 13 Sep 2019 SAD MFCC X-vector extraction PLDA scoring AHC SAD neural network X-vector neural network Same/Diff covariance matrices Diarization result (a) X-vector clustering-based method Log-Mel Joint speech activity detection of all speakers EEND neural network Diarization result (b) EEND method

show abstract

“…It is compared with two conventional methods and two simple extension of the conventional method for block processing: (i) PIT and (ii) RSAN applied to the whole mixture (called PIT batch and RSAN batch, hereafter), extensions of RSAN to perform diarization in (iii) block-online and (iv) blockoffline manners. These simple extensions are 2-stage methods similar to the conventional methods in [1,[10][11][12][13], which, based on NN, first separate the speakers, estimate associated speaker embedding vectors, and then cluster the vectors to estimate the correct association of speaker identity information among blocks. The methods (iii) and (iv) are referred to as online and offline clustering, hereafter.…”

Section: Methodsmentioning

confidence: 99%

All-neural Online Source Separation, Counting, and Diarization for Meeting Analysis

Neumann

Kinoshita

Delcroix

et al. 2019

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Automatic meeting analysis comprises the tasks of speaker counting, speaker diarization, and the separation of overlapped speech, followed by automatic speech recognition. This all has to be carried out on arbitrarily long sessions and, ideally, in an online or block-online manner. While significant progress has been made on individual tasks, this paper presents for the first time an all-neural approach to simultaneous speaker counting, diarization and source separation. The NN-based estimator operates in a block-online fashion and tracks speakers even if they remain silent for a number of time blocks, thus learning a stable output order for the separated sources. The neural network is recurrent over time as well as over the number of sources. The simulation experiments show that state of the art separation performance is achieved, while at the same time delivering good diarization and source counting results. It even generalizes well to an unseen large number of blocks.

show abstract

BUT System for DIHARD Speech Diarization Challenge 2018

Cited by 46 publications

References 14 publications

The Second DIHARD Diarization Challenge: Dataset, Task, and Baselines

The Second DIHARD Diarization Challenge: Dataset, Task, and Baselines

End-to-End Neural Speaker Diarization with Self-Attention

All-neural Online Source Separation, Counting, and Diarization for Meeting Analysis

Contact Info

Product

Resources

About