New approaches to audio-visual segmentation of TV news for automatic topic retrieval

Abstract

This paper presents two new real-time approaches to segmentation of TV news shows into topics. The goal of this research work is the high precision retrieval of topics from TV news. For that purpose, the detection of correct topic boundaries is of great importance. We introduce a stochastic and a rule-based topic model based on HMMs. The former combines features from the visual as well as from the audio channel of the news show, whereas the latter uses the video channel only. They are compared to the detection of topics using only the audio channel, which is common for many other approaches. The paper contains the following innovations: 1) The detected segment boundaries correspond directly to topics and not to video or audio cuts, as most other segmentation methods. 2) An advanced stochastic topic model is introduced that uses audio as well as video features.3) The introduced HMM-based approaches both outperform the audio-based approach. One algorithm has a very good topic boundary detection rate, whereas the other minimizes the number of wrongly inserted boundaries without missing too many real boundaries.

<title>Logical unit and scene detection: a comparative survey</title>

“…The internal structure of video has been modeled in literature to facilitate the detection of logical unit boundaries. Iurgel [14] proposed a video model for news shows, part of which is shown in Fig. 9.…”

“…They consist of topic units with compilation and continuity cutting. Specific news models have been used to capture this type of structure [14,24]. Training and instructional videos focus on teaching and often an instructor is audibly and visibly addressing the audience of the video or interacting with an audience visible in the video.…”

“…A continuation identifier, on the other hand, is exploited in the camera work rule, which states if three or more shots have the same camera work (other than still shot) they belong to the same logical unit. In [14] several approaches were presented for news story segmentation, one of which was based on rules. First an HMM was employed to classify shots into one of six classes: begin, end, newscaster, report, interview, and weather forecast.…”

“…Approaches for news story segmentation usually exploit the fact that an introductory shot, usually an anchor shot, is followed by report shots for each news story. A very detailed model for news is proposed in [14]. It uses six content classes which are begin, end, newscaster, interview, report and weather forecast.…”

Detecção de cenas em segmentos semanticamente complexos

“…Esse fato pode ser explicado pela capacidade da parte visual de transmitir uma grande parte da semântica latente presente em um vídeo, comprovada em vários trabalhos (Fabro e Böszörmenyi, 2013;Coimbra, 2011;Iurgel et al, 2001). …”