Automatic video annotation via Hierarchical Topic Trajectory Model considering cross-modal correlations

Abstract

We propose a new statistical model, named Hierarchical Topic Trajectory Model (HTTM), for acquiring a dynamically changing topic model that represents the relationship between video frames and associated text labels. Model parameter estimation, annotation and retrieval can be executed within a unified framework with a few computation. It is also easy to add new modals such as audio signal and geotags. Preliminary experiments on video annotation task with manually annotated video dataset indicate that our proposed method can improve the annotation accuracy.

Comparative analysis on cross-modal information retrieval: A review

“…Thus, it is important to have precise spectrum representation with maximum information gain and no redundancy. For example, in the case of image and text, images can be Image retrieval in different categories 2014, 2019 [38], [39] 24 distinct category image-text retrieval 2012 [40] Disaster and emergency management 2016 [25] image-text retrieval in various categories 2017, 2018 [41], [42] 10 I, T, V Image-text and video-text retrieval in multiple categories 2015 [43] Video, image and text retrieval in video lectures 2014 [44] 11 T, V Multiple concepts' video annotation 2011 [45] cooking activities' video annotation, videos' temporal activity localization evaluation, personal videos' annotation 2019 [46] Cooking recipe retrieval 2019 [28] represented in spatial or spectral while the text is symbolic and dependent upon grammar rules and cultural norms [2].…”