A unified approach to content-based indexing and retrieval of digital videos from television archives

Abstract

This work addresses the development of a unified approach to content-based indexing and retrieval of digital videos from television archives. The proposed approach has been designed to deal with arbitrary television genres, making it suitable for various applications. To achieve this goal, the main steps of a content-based video retrieval system are addressed in this work, namely: video segmentation, key-frame extraction, content-based video indexing and the video retrieval operation itself. Video segmentation is addressed as a typical TV broadcast structuring problem, which consists in automatically determining the boundaries of each broadcasted program (like movies, news, among others) and inter-program (for instance, commercials). Specifically, to segment the videos, Electronic Program Guide (EPG) metadata is combined with the detection of two special cues, namely, audio cuts (silence) and dark monochrome frames. On the other hand, a color histogram-based approach performs key-frame extraction. Video indexing and retrieval are accomplished by using hashing and k-d tree methods, while visual signatures containing color, shape and texture information are estimated for the key-frames, by using image and frequency domain techniques. Experimental results with the dataset of a multimedia information system especially developed for managing television broadcast archives demonstrate that our approach works efficiently, retrieving videos in 0.16 seconds on average and achieving recall, precision and F1 measure values, as high as 0.76, 0.97 and 0.86 respectively.

Video Indexing and Retrieval based on Key Frame Extraction

“…Related works on video retrieval methods are vast, and it is not our goal to cover them here completely. Souza et al [4] present a method for content-based indexing and retrieval of TV video sequences. The main steps of this method include video boundary detection, key frame selection, and content-based video retrieval, where hashing and k-d tree methods are employed to accomplish video indexing and retrieval.…”

<b>Semiodiscursive analysis of TV newscasts based on data mining and image processing

“…To compute this visual signature, we use the approach proposed by Souza, Pádua, Nunes, Assis, and Silva (2014), which obtains a visual signature containing 79 components representing each face image (54 refer to color, 18 refer to texture, and seven refer to shape positions).…”

Selecting Video Key Frames Based on Relative Entropy and the Extreme Studentized Deviate Test

“…Generally, shot-based key frame selection methods are the main option for practical use, due to their simple mechanism and good running efficiency [8,19]. It is worth pointing out that the distance between video frames plays a vital role in shot-based video key frame extraction [20].…”

“…In general, due to its fast running and easy implementation, shot-based key frame extraction is probably the most used framework for video summarization in practice [5,7,8]. Needless to say, dealing with video data of any kind is valuable for real applications [9,10].…”