Semantic concept based video retrieval using convolutional neural network

Abstract

Retrieval of videos efficiently and effectively has become a challenging issue nowadays and dealing with multi-concept videos is the center of focus. The aim of the work presented here is to propose an improved semantic concept-based video retrieval method using a novel ranked intersection filtering technique and a foreground driven concept co-occurrence matrix. In the proposed ranked intersection filtering technique, an intersection of ranked concept probability scores is taken from key-frames associated with a query shot to identify concepts to be used in retrieval. Convolutional neural network is used as a baseline. The proposed method is implemented using a classifier built with a fusion of asymmetrically trained deep CNNs to deal with data imbalance problem, a novel foreground driven concept co-occurrence matrix to exploit concept co-occurrence information and a ranked intersection filtering approach. Performance is evaluated by a measure, mean average precision on TRECVID multi-label dataset. The results are compared with state-of-the-art other existing methods in its class and shown its superiority.

Deep Hybrid Neural Network With Attention Mechanism for Video Hash Retrieval Method

“…Paper [15] proposed a nested invariant pool (NIP) method to obtain compact and discriminative convolutional neural network video retrieval descriptors, which mainly acts on the middle feature mapping of CNN in a nested manner, reducing the feature size of CNN, boosting the geometric invariance of depth descriptors, and significantly improving the performance of video retrieval. In [16] uses a classififier fusing asymmetric training depth CNNs to deal with data imbalance to achieve effificient video retrieval. The dual encoding network proposed in reference [17] encodes inputs in a similar manner.…”

A novel multilabel video retrieval method using multiple video queries and deep hash codes

“…Finally, the classification scores of each shot are fused to obtain the final class score. Similar studies are based on the shot boundary detection and keyframe extraction technique 11,12 . In such studies, video semantics are only considered on the frame level.…”

“…Similar studies are based on the shot boundary detection and keyframe extraction technique. 11,12 In such studies, video semantics are only considered on the frame level. There have been various studies considering spatio-temporal characteristics.…”