Speaker Change Detection Using Fundamental Frequency with Application to Multi-talker Segmentation

Abstract

This paper shows that time varying pitch properties can be used advantageously within the segmentation step of a multi-talker diarization system. First a study is conducted to verify that changes in pitch are strong indicators of changes in the speaker. It is then highlighted that an individual's pitch is smoothly varying and therefore can be predicted by means of a Kalman filter. Subsequently it is shown that if the pitch is not predictable then this is most likely due to a change in the speaker. Finally, a novel system is proposed that uses this approach of pitch prediction for speaker change detection. This system is then evaluated against a commonly used MFCC segmentation system. The proposed system is shown to increase the speaker change detection rate from 43.3% to 70.5% on meetings in the AMI corpus. Therefore there are two equally weighted contributions in this paper: 1. We address the question of whether a change in pitch is a reliable estimator of a speaker change in multi-talk meeting audio. 2. We develop a method to extract such speaker changes and test them on a widely available meeting corpus.

Overlapping Speaker Segmentation Using Multiple Hypothesis Tracking of Fundamental Frequency

“…To explore this question on the AMI corpus [36], pitch estimates were calculated using the method of [21] applied to the IHM mixed-down stream of 16 AMI meetings. A Kalman filter [37] was used to track the pitch of the IHM mixed-down single channel stream as proposed in [38]. The Kalman track relies on the smooth variation of a speaker's pitch due to physiological constraints [39].…”

“…Speaker change detection is achieved by exploiting the temporal variations in the pitch. To accomplish this the method of [38], that operates using a single track, is utilised here. It was shown in Section III-C that multiple tracks can be generated for the same speaker.…”

Multiple Hypothesis Tracking for Overlapping Speaker Segmentation

“…To obtain the full segmentation it is also necessary to detect speaker changes, ct, when the onset of a new speaker happens after the end-point of the previous speaker. To accomplish this the authors' previous method, that operates using a single track, is further developed here [21]. Speaker change detection is achieved by exploiting the temporal variations in the pitch.…”

“…(d) Segments of overlapping speech are identified as the onsets and end-points of multiple, uncorrelated pitch tracks. (e) The complete segmentation is obtained from the union of overlapping speech onsets with the speaker changes detected based on a model of the temporal variation of pitch [21].…”

Multichannel Overlapping Speaker Segmentation Using Multiple Hypothesis Tracking Of Acoustic And Spatial Features

“…A number of approaches were proposed to solve the problem of speaker segmentation. Most of these methods rely on features that fall into three separate categories: acoustic features [9,10], spatial features [11] and linguistic features [12]. More recently data driven, deep learning approaches have become popular [13][14][15].…”