Temporal Video Segmentation by Graph Partitioning

Abstract

Abstract-A novel temporal video segmentation method that, in addition to abrupt cuts, can detect with very high accuracy gradual transitions such as dissolves, fades and wipes is proposed. The method relies on evaluating mutual information between multiple pairs of frames within a certain temporal frame window. This way we create a graph where the frames are nodes and the measures of similarity correspond to the weights of the edges. By finding and disconnecting the weak connections between nodes we separate the graph to subgraphs ideally corresponding to the shots. Experiments on TRECVID2004 video test set containing different types of shot transitions and significant object and camera motion inside the shots prove that the method is very efficient.

Video Analysis Based on Mutual Information

“…For a video it is calculated for two successive frames for each RGB or YUV component separately [10]. For the calculation we need matrices C (j) show the total number of pixels with the intensity level i in the frame f t , respectively with the level j in the frame f t+1 divided by number of pixels in the frame.…”

“…For the calculation we need matrices C (j) show the total number of pixels with the intensity level i in the frame f t , respectively with the level j in the frame f t+1 divided by number of pixels in the frame. The mutual information I K t,t+1 for each RGB or YUV component is expressed by [10]:…”

“…Therefore, the correct detection of abrupt shot transitions is a very important task in the video segmentation and this paper is only focused on the detection of an abrupt cut. Our aim was to improve the existing method proposed by Cernekova [10]. This method uses the mutual information, but it is very sensitive to object or camera motion.…”

Prediction Methods for MPEG-4 and H.264 Video Transmission

“…The motivation to develop the MPEG-7 standard was based on the fact, that the potential of the digital video has given to the producers the editing tools to manipulate all the film data faster and with better accuracy. Therefore the standardization of features became more and more important [11]. The MPEG-7 standard is different from the older standards, which represent the content, while the MPEG-7 standard represents information about the content.…”

“…For this purpose, the standard defines syntax and semantics for describing multimedia contents. The purpose of the standard is to increase the interoperability between different applications and devices that work with audio-visual content descriptions [11].…”