Recent Advances in Video Content Analysis: From Visual Features to Semantic Video Segments

Abstract

This paper addresses the problem of automatically partitioning a video into semantic segments using visual low-level features only. Semantic segments may be understood as building content blocks of a video with a clear sequential content structure. Examples are reports in a news program, episodes in a movie, scenes of a situation comedy or topic segments of a documentary. In some video genres like news programs or documentaries, the usage of different media (visual, audio, speech, text) may be beneficial or is even unavoidable for reliably detecting the boundaries between semantic segments. In many other genres, however, the pay-off in using different media for the purpose of high-level segmentation is not high. On the one hand, relating the audio, speech or text to the semantic temporal structure of video content is generally very difficult. This is especially so in "acting" video genres like movies and situation comedies. On the other hand, the information contained in the visual stream of these video genres often seems to provide the major clue about the position of semantic segments boundaries. Partitioning a video into semantic segments can be performed by measuring the coherence of the content along neighboring video shots of a sequence. The segment boundaries are then found at places (e.g., shot boundaries) where the values of content coherence are sufficiently low. On the basis of two state-of-the-art techniques for content coherence modeling, we illustrate in this paper the current possibilities for detecting the boundaries of semantic segments using visual low-level features only.

Система Ідентифікації Оригіналу Відео За Його Фрагментом З Використанням Згорткових Нейронних Мереж

“…Кластеризація завжди є рішенням для скорочення та організації вмісту відео, окрім цього, забезпечує ефективну схему індексації для пошуку відео, оскільки подібні кадри згруповані в одному кластері. Верхній рівень кластеризації згрупований за властивостями колірної схеми, тоді як нижній рівень -за функціями руху [5].…”

A metadata classification schema for semantic content analysis of videos

“…Video semantic content analysis and the use of structural metadata for searching image and video libraries has in recent years become the subject of intense research, with a large and expanding bibliography (see, for example, Kimitrova et al. , 2000; Hanjalic et al. , 2001; and papers in Panchanathan & Liu, 1996; Del Bimbo et al.…”