Broadcast News Story Segmentation Using Conditional Random Fields and Multimodal Features

Abstract

SUMMARYIn this paper, we propose integration of multimodal features using conditional random fields (CRFs) for the segmentation of broadcast news stories. We study story boundary cues from lexical, audio and video modalities, where lexical features consist of lexical similarity, chain strength and overall cohesiveness; acoustic features involve pause duration, pitch, speaker change and audio event type; and visual features contain shot boundaries, anchor faces and news title captions. These features are extracted in a sequence of boundary candidate positions in the broadcast news. A linear-chain CRF is used to detect each candidate as boundary/non-boundary tags based on the multimodal features. Important interlabel relations and contextual feature information are effectively captured by the sequential learning framework of CRFs. Story segmentation experiments show that the CRF approach outperforms other popular classifiers, including decision trees (DTs), Bayesian networks (BNs), naive Bayesian classifiers (NBs), multilayer perception (MLP), support vector machines (SVMs) and maximum entropy (ME) classifiers.

Topic segmentation in ASR transcripts using bidirectional RNNS for change detection

“…Previous works on topic segmentation have adopted sequence modelling approaches based on Hidden Markov Models (HMM) [13,17,22,29], and also discriminatively trained models including Conditional Random Fields (CRF) [30], deep feed forward neural network with HMM [29] and Support Vector Machines with sliding windows [26]. However, RNNs are better than fixed size windows and HMMs at exploiting contextual information [27].…”

A deep neural network approach for sentence boundary detection in broadcast news

“…Nicola et al [2] studied various lexical features, including language model features, sentence length features and syntax features, on different genres ranging from formal newspaper text to informal, dictated messages, and from written text to spoken transcript. Recent efforts have shown that speech prosody, especially pause and pitch related features, are informative indicators for structural events [1, 3,4,5] including sentence boundaries [6,7,8,9]. Research has shown that a decision tree (DT) model learned from prosodic features can achieve comparable performance with that learned from lexical features.…”