Video Captioning via Hierarchical Reinforcement Learning

Abstract

Video captioning is the task of automatically generating a textual description of the actions in a video. Although previous work (e.g. sequence-to-sequence model) has shown promising results in abstracting a coarse description of a short video, it is still very challenging to caption a video containing multiple fine-grained actions with a detailed description. This paper aims to address the challenge by proposing a novel hierarchical reinforcement learning framework for video captioning, where a highlevel Manager module learns to design sub-goals and a low-level Worker module recognizes the primitive actions to fulfill the sub-goal. With this compositional framework to reinforce video captioning at different levels, our approach significantly outperforms all the baseline methods on a newly introduced large-scale dataset for fine-grained video captioning. Furthermore, our non-ensemble model has already achieved the state-of-the-art results on the widelyused MSR-VTT dataset. Caption: A person sits on a bed and puts a laptop into a bag.The person stands up, puts the bag on one shoulder, and walks out of the room. Caption #1: A woman offers her dog some food.Caption #2: A woman is eating and sharing food with her dog. Caption #3: A woman is sharing a snack with a dog.

Beyond the feedforward sweep: feedback computations in the visual cortex

“…Initial work has already demonstrated the benefits of combining reinforcement learning with RNNs to play Atari ® games 145 . Promising results have also been obtained for visual tracking, 146,147 face recognition, 148 action recognition, 149,150 video captioning, 151 color enhancement, 152 and object detection 153,154 …”

“…Initial work has already demonstrated the benefits of combining reinforcement learning with RNNs to play Atari R games. 145 Promising results have also been obtained for visual tracking, 146,147 face recognition, 148 action recognition, 149,150 video captioning, 151 color enhancement, 152 and object detection. 153,154 Another approach to learning structure in the visual world, which does not use explicit labeled examples or a teacher and provides direct rewards/punishment for specific actions, is based on the intuition that predicting what will happen next may be an important principle of computation in the brain.…”

Transferable Representation Learning in Vision-and-Language Navigation

“…Vision-and-Language Grounding There is much prior work in the intersection of computer vision and natural language processing [42,23,27,21]. A highly related class of tasks centers around generating captions for images and videos [12,13,37,38,44]. In Visual Question Answering [3,43] and Visual Dialog [9], models generate single-turn and multi-turn responses by co-grounding vision and language.…”

Low-Rank HOCA: Efficient High-Order Cross-Modal Attention for Video Captioning

“…(Shen et al, 2017;Gan et al, 2017) adopt multi-label learning with weak supervision to extract semantic features of video data. (Wang et al, 2018b) proposes optimizing the metrics directly with hierarchical reinforcement learning. extracts five types of features to develop the multimodal video captioning method and achieves promising results.…”