SpeechFind: advances in spoken document retrieval for a National Gallery of the Spoken Word

Abstract

Abstract-Advances in formulating spoken document retrieval for a new National Gallery of the Spoken Word (NGSW) are addressed. NGSW is the first large-scale repository of its kind, consisting of speeches, news broadcasts, and recordings from the 20th century. After presenting an overview of the audio stream content of the NGSW, with sample audio files from U.S. Presidents from 1893 to the present, an overall system diagram is proposed with a discussion of critical tasks associated with effective audio information retrieval. These include advanced audio segmentation, speech recognition model adaptation for acoustic background noise and speaker variability, and information retrieval using natural language processing for text query requests that include document and query expansion. For segmentation, a new evaluation criterion entitled fused error score (FES) is proposed, followed by application of the CompSeg segmentation scheme on DARPA Hub4 Broadcast News (30.5% relative improvement in FES) and NGSW data. Transcript generation is demonstrated for a six-decade portion of the NGSW corpus. Novel model adaptation using structure maximum likelihood eigenspace mapping shows a relative 21.7% improvement. Issues regarding copyright assessment and metadata construction are also addressed for the purposes of a sustainable audio collection of this magnitude. Advanced parameter-embedded watermarking is proposed with evaluations showing robustness to correlated noise attacks. Our experimental online system entitled "SpeechFind" is presented, which allows for audio retrieval from a portion of the NGSW corpus. Finally, a number of research challenges such as language modeling and lexicon for changing time periods, speaker trait and identification tracking, as well as new directions, are discussed in Manuscript

An overview of automatic speaker diarization systems

“…(see Fig. 6), and the inclusion in complete retrieval systems such as Rough 'n' Ready [55] and SpeechFind [56] allow users to see the current speaker information, understand the general flow of speakers throughout the broadcast, or search for a particular speaker within the audio. Experiments are also underway to ascertain if additional tasks, such as the process of annotating data, can be facilitated using diarization output.…”

Computationally Efficient and Robust BIC-Based Speaker Segmentation

“…Previous experiment demonstrates that under-segmentation, caused by a high number of miss detections, is more cumbersome to remedy than over-segmentation caused by a high number of false alarms [12], [13], [15], [16], [23], [40]. For example, over-segmentation could be alleviated by clustering and/or merging.…”

“…The window size is also set equal to r taking into consideration as many data as possible. When more data are available, more accurate Gaussian models are built, since BIC behaves better for large windows, whereas short changes are not easily detectable by BIC [12], [16]. Moreover, it was shown in [22], that the bigger the window size, the better the performance.…”

Robust Unsupervised Speaker Segmentation for Audio Diarization

“…Having segmented speech regions, it is also often necessary to segment these further in terms of homogeneous speaker turns. In addition to improving ASR systems, speaker turn information can be helpful for speaker adaptation in rich transcription of videos and meetings (Bonastre et al, 2000) and for content based audio classification and retrieval (Hansen et al, 2005) which have a wide range of applications in the entertainment industry, audio archive management, surveillance, etc. Audio segmentation would also be an important tool in summarizing meetings, which has recently gained a lot of interest in the research community.…”