VISCOM: A robust video summarization approach using color co-occurrence matrices

Cirne, Marcos Vinícius Mussel; Pedrini, Hélio

doi:10.1007/s11042-016-4300-7

Cited by 28 publications

(3 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that, within this process, labels were not necessary; thus, the DNS approach may operate completely unsupervised. In [22], a unique video-summarizing technique called VISCOM was introduced, which is based on the color occurrence matrices from the video, which were then utilized to characterize each video frame. Then, from the most informative frames of the original video, a summary was created.…”

Section: Related Workmentioning

confidence: 99%

Video Summarization Based on Feature Fusion and Data Augmentation

Psallidas,

Spyrou

2023

Computers

View full text Add to dashboard Cite

During the last few years, several technological advances have led to an increase in the creation and consumption of audiovisual multimedia content. Users are overexposed to videos via several social media or video sharing websites and mobile phone applications. For efficient browsing, searching, and navigation across several multimedia collections and repositories, e.g., for finding videos that are relevant to a particular topic or interest, this ever-increasing content should be efficiently described by informative yet concise content representations. A common solution to this problem is the construction of a brief summary of a video, which could be presented to the user, instead of the full video, so that she/he could then decide whether to watch or ignore the whole video. Such summaries are ideally more expressive than other alternatives, such as brief textual descriptions or keywords. In this work, the video summarization problem is approached as a supervised classification task, which relies on feature fusion of audio and visual data. Specifically, the goal of this work is to generate dynamic video summaries, i.e., compositions of parts of the original video, which include its most essential video segments, while preserving the original temporal sequence. This work relies on annotated datasets on a per-frame basis, wherein parts of videos are annotated as being “informative” or “noninformative”, with the latter being excluded from the produced summary. The novelties of the proposed approach are, (a) prior to classification, a transfer learning strategy to use deep features from pretrained models is employed. These models have been used as input to the classifiers, making them more intuitive and robust to objectiveness, and (b) the training dataset was augmented by using other publicly available datasets. The proposed approach is evaluated using three datasets of user-generated videos, and it is demonstrated that deep features and data augmentation are able to improve the accuracy of video summaries based on human annotations. Moreover, it is domain independent, could be used on any video, and could be extended to rely on richer feature representations or include other data modalities.

show abstract

Section: Related Workmentioning

confidence: 99%

Video Summarization Based on Feature Fusion and Data Augmentation

Psallidas,

Spyrou

2023

Computers

View full text Add to dashboard Cite

show abstract

“…To address these problems, shot boundary detection [10], [11], [12] and meta-data, such as audio signal [13], [14], transcripts [15], [16] approaches are proposed to summarize lecture videos. Subudhi et al [10] compared frames histograms to detect shot transition and calculate edge function based on three contrast features to estimate content and non-content frames.…”

Section: Related Workmentioning

confidence: 99%

“…Mohanta et al [11] utilized local and global features to detect and classify shot boundary and then a multilayer perceptron network was applied to detect no change, abrupt change and gradual change frames. Cirne et al [12] described a video frame by means of color co-occurrence matrices, then normalized sum of squared differences was used to detect shot boundary. He et al [14] exploited spoken text, pitch and pause information in audio signal to show that these changes under various speaking conditions.…”

Section: Related Workmentioning

confidence: 99%

Spatio-Temporal Residual Networks for Slide Transition Detection in Lecture Videos

2019

KSII TIIS

View full text Add to dashboard Cite

In this paper, we present an approach for detecting slide transitions in lecture videos by introducing the spatio-temporal residual networks. Given a lecture video which records the digital slides, the speaker, and the audience by multiple cameras, our goal is to find keyframes where slide content changes. Since temporal dependency among video frames is important for detecting slide changes, 3D Convolutional Networks has been regarded as an efficient approach to learn the spatio-temporal features in videos. However, 3D ConvNet will cost much training time and need lots of memory. Hence, we utilize ResNet to ease the training of network, which is easy to optimize. Consequently, we present a novel ConvNet architecture based on 3D ConvNet and ResNet for slide transition detection in lecture videos. Experimental results show that the proposed novel ConvNet architecture achieves the better accuracy than other slide progression detection approaches.

show abstract

Hybrid feature extraction and optimized deep convolutional neural network based video shot boundary detection

Mishra

2022

Concurrency and Computation

View full text Add to dashboard Cite

Summary This paper presents a hybrid feature extraction and optimized deep learning model for the effective shot boundary detection (SBD). A cross‐guided bilateral filtering technique is used for pre‐processing. A hybrid fuzzy histogram with dual tree complex wavelet transform (FH‐DTCWT) is developed for the feature extraction to extract the visual features from the pre‐processed frames of the block. Candidate segment selection process is performed to identify the non‐boundary frames from the extracted features to improve the accuracy. Continuity matrix is created using the possible transition frames to verify that the frames are in sequential order without any gap. Classification of transition types are performed by the proposed optimized deep convolutional neural network. Optimized deep learning (DCNN‐RBESO) model is the combination of deep convolutional neural network (DCNN) and Rider bald eagle search optimization (RBESO) algorithm. To update the weight of DCNN model in learning, RBESO is utilized. MATLAB tool is used to develop the proposed SBD model. The experimental analysis is carried out for the proposed model using TREC Video Retrieval Evaluation (TRECVID) and VideoSeg datasets. The proposed model outperformed in the detection of shot boundaries in terms of precision, recall, and F1‐score.

show abstract

VISCOM: A robust video summarization approach using color co-occurrence matrices

Cited by 28 publications

References 48 publications

Video Summarization Based on Feature Fusion and Data Augmentation

Video Summarization Based on Feature Fusion and Data Augmentation

Spatio-Temporal Residual Networks for Slide Transition Detection in Lecture Videos

Hybrid feature extraction and optimized deep convolutional neural network based video shot boundary detection

Contact Info

Product

Resources

About