Meta Learning for Task-Driven Video Summarization

Abstract

Existing video summarization approaches mainly concentrate on sequential or structural characteristic of video data. However, they do not pay enough attention to the video summarization task itself. In this paper, we propose a meta learning method for performing task-driven video summarization, denoted by MetaL-TDVS, to explicitly explore the video summarization mechanism among summarizing processes on different videos. Particularly, MetaL-TDVS aims to excavate the latent mechanism for summarizing video by reformulating video summarization as a meta learning problem and promote generalization ability of the trained model. MetaL-TDVS regards summarizing each video as a single task to make better use of the experience and knowledge learned from processes of summarizing other videos to summarize new ones. Furthermore, MetaL-TDVS updates models via a two-fold back propagation which forces the model optimized on one video to obtain high accuracy on another video in every training step. Extensive experiments on benchmark datasets demonstrate the superiority and better generalization ability of MetaL-TDVS against several state-of-the-art methods.

Machine learning for perovskite materials design and discovery

“…Therefore, developing faster, more accurate, advanced, and intelligent learning algorithms to deal with the challenge of insufficient data would be very indispensable, especially when most data in publications about ML in perovskite materials belong to small samples. A common method to deal with small samples is meta-learning, that is, learning knowledge within or across a specific field 136,137 . The development of new technologies such as neural Turing machines 138 and imitation learning 139 could make it possible.…”

Vector ordering and regression learning‐based ranking for dynamic summarisation of user videos

“…where d i j are exactly those used in (28). Just like the previous case, a two-sample t-test is performed comparing the populations of score, s(v i ) and s(u i ), ∀i.…”

Query Twice: Dual Mixture Attention Meta Learning for Video Summarization

“…The F1-score results can reflect that our attention mechanism with meta learning can better predict importance scores. [32] 45.3 58.2 SUM-GAN [21] 41.7 54.3 Cycle-SUM [35] 41.9 57.6 DR-DSN [39] 42.1 58.1 MetaL-TDVS [17] 44.1 58.2 ACGAN [11] 46.0 58.5 CSNet [13] 51.3 58.8 M-AVS [12] 44.4 61.0…”

“…Recently, meta learning methods have been applied in a few video analysis tasks. Especially in video summarization, Li et al [17] proposed a meta learning method that explores the video summarization mechanism among summarizing processes on different videos.…”

“…The compared methods can be mainly categorized into LSTM, GAN, Attention, and meta learning models. M-AVS [12] and ACGAN [11] are based-on attention models and MetaL-TDVS [17] is based on meta learning. It can be seen that DMASum outperforms other approaches on both datasets consistently.…”