Auto encoder with mode‐based learning for keyframe extraction in video summarization

Abstract

The exponential increase in video consumption has created new difficulties for browsing and navigating through video more effectively and efficiently. Researchers are interested in video summarization because it offers a brief but instructive video version that helps users and systems save time and effort when looking for and comprehending relevant content. Key frame extraction is a method of video summarization that only chooses the most important frames from a given video. In this article, a novel supervised learning method ‘TC‐CLSTM Auto Encoder with Mode‐based Learning’ using temporal and spatial features is proposed for automatically choosing keyframes or important sub‐shots from videos. The method was able to achieve an average F‐score of 84.35 on TVSum dataset. Extensive tests on benchmark data sets show that the suggested methodology outperforms state‐of‐the‐art methods.