Multi-label semantic concept detection in videos using fusion of asymmetrically trained deep convolutional neural networks and foreground driven concept co-occurrence matrix

Janwe, Nitin; Bhoyar, Kishor K.

doi:10.1007/s10489-017-1033-x

Cited by 14 publications

(12 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A fusion of multiple asymmetrically trained CNNs, a novel approach proposed by us in [1], is used for classifier implementation. The approach of asymmetric training has been very effective while dealing with imbalanced datasets.…”

Section: Building Cnn Classifiermentioning

confidence: 99%

“…If the same classifier is used to train both strong and weak classes, it becomes difficult to achieve better detection rate for both categories of classes. Therefore, it was proposed in [1] to use an approach of a fusion of asymmetric trained deep CNNs to build a classifier to deal with imbalanced dataset samples. Here, two separate CNNs with the same architecture are trained independently (asymmetrically) with the same dataset to classify strong and weak classes respectively as shown in Fig.…”

Section: Building Cnn Classifiermentioning

confidence: 99%

“…The work in this paper is an extension of [1]. Here, we addressed the issue of semantic video retrieval and proposed and implemented a novel approach for retrieval with improved performance.…”

Section: Introductionmentioning

confidence: 99%

“…Here, we addressed the issue of semantic video retrieval and proposed and implemented a novel approach for retrieval with improved performance. In [1], we have shown that an efficient concept classifier can be implemented using deep convolutional neural network (CNN) by extracting semantic features from video. In this work, the same approach is used for implementing a classifier.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Semantic concept based video retrieval using convolutional neural network

Janwe¹,

Bhoyar

2019

SN Appl. Sci.

View full text Add to dashboard Cite

Retrieval of videos efficiently and effectively has become a challenging issue nowadays and dealing with multi-concept videos is the center of focus. The aim of the work presented here is to propose an improved semantic concept-based video retrieval method using a novel ranked intersection filtering technique and a foreground driven concept co-occurrence matrix. In the proposed ranked intersection filtering technique, an intersection of ranked concept probability scores is taken from key-frames associated with a query shot to identify concepts to be used in retrieval. Convolutional neural network is used as a baseline. The proposed method is implemented using a classifier built with a fusion of asymmetrically trained deep CNNs to deal with data imbalance problem, a novel foreground driven concept co-occurrence matrix to exploit concept co-occurrence information and a ranked intersection filtering approach. Performance is evaluated by a measure, mean average precision on TRECVID multi-label dataset. The results are compared with state-of-the-art other existing methods in its class and shown its superiority.

show abstract

Section: Building Cnn Classifiermentioning

confidence: 99%

Section: Building Cnn Classifiermentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Semantic concept based video retrieval using convolutional neural network

Janwe¹,

Bhoyar

2019

SN Appl. Sci.

View full text Add to dashboard Cite

show abstract

“…Nowadays, the research object of digital geometry processing is gradually crossing from low-level geometric attributes to high-level semantic attributes. The symmetry of a three-dimensional (3D) geometric model is an important bridge between low-level geometric information and high-level semantic information [1,2]. Symmetry analysis is an important problem in the field of geometric processing, which is widely used in the segmentation, editing, and retrieval of 3D geometric models.…”

Section: Introductionmentioning

confidence: 99%

Research on the Detection Method of Implicit Self Symmetry in a High-Level Semantic Model

Wang

2019

Symmetry

View full text Add to dashboard Cite

In order to improve the accuracy of semantic model intrinsic detection, a skeleton-based high-level semantic model intrinsic self-symmetry detection method is proposed. The semantic analysis of the model set is realized by the uniform segmentation of the model within the same style, the component correspondence of the model between different styles, and the shape content clustering. Based on the results of clustering analysis, for a given three-dimensional (3D) point cloud model, according to the curve skeleton, the skeleton point pairs reflecting the symmetry between the model surface points are obtained by the election method, and the symmetry is extended to the model surface vertices according to these skeleton point pairs. With the help of skeleton, the symmetry of the point cloud model is obtained, and then the symmetry region of point cloud model is obtained by the symmetric correspondence matrix and spectrum method, so as to realize the intrinsic symmetry detection of the model. The experimental results show that the proposed method has the advantages of less time, high accuracy, and high reliability.

show abstract

A novel multilabel video retrieval method using multiple video queries and deep hash codes

Akbacak

2022

Concurrency and Computation

View full text Add to dashboard Cite

Most videos consist of multiple clips. Namely, videos usually have multiple spatio-temporal information. However, existing video datasets are of single video clips.Additionally, content-based video retrieval methods have been implemented based on a single query clip. However, a single query may not be sufficient for better expressing intention concerning the query. Besides, it narrows the capability to search for multiple semantics. How to deal with retrieving videos having multiple clips has not been adequately investigated. This study proposes a novel multilabel video retrieval method that utilizes multiple video clips as queries. Whatever the number of queries, it transforms a multi-query video retrieval into a single-query video retrieval. The method is independent of both the video features and distance metric types. However, we propose a deep video hashing method to achieve speed and efficiency. Experiments performed on three prepared multilabel video datasets have proven the effectiveness of the proposed method. The source codes of the study are public. K E Y W O R D Scontent-based video retrieval, multi-query optimal point, multiple video queries, objective space, video hashing INTRODUCTIONContent-based video retrieval (CBVR) is the process of retrieving videos from a video database by querying them based on visual and temporal content. The visual content is usually an object, face, color, texture, or scene at the frame level. On the other hand, the temporal content is an action type at the video level. Videos frames, each of which is an image, usually have multilabel characters. At the video level, the temporal relation between frames also expresses a piece of semantic information. 1,2 It is called action type. An activity on a football field other than playing football tends to the football semantic in terms of frame-level. However, at the video level, the action type is not football.Neglecting temporal dependencies is sometimes an obligation, but sometimes a deficiency. [3][4][5] For face or object retrieval, for example, it is not necessitated. [6][7][8][9] However, some CBVR studies fundamentally ignore temporal properties. Janwe et al. used video segmentation to get the video shots. 10 From each shot, keyframes that adequately represent the semantic meaning of each shot are extracted. Then the keyframe features are extracted and are trained in a multi-stream CNN classifier. Finally, the classification scores of each shot are fused to obtain the final class score. Similar studies are based on the shot boundary detection and keyframe extraction technique. 11,12 In such studies, video semantics are only considered on the frame level. There have been various studies considering spatio-temporal characteristics. [13][14][15][16][17] On the other hand, few studies only consider actions but ignore pixel-level features. 14,18 The most concise video scene with at least one semantic meaning at the video level is called a clip. Naturally, videos made up of clips are multilabel. The use of a single video clip qu...

show abstract

Multi-label semantic concept detection in videos using fusion of asymmetrically trained deep convolutional neural networks and foreground driven concept co-occurrence matrix

Cited by 14 publications

References 31 publications

Semantic concept based video retrieval using convolutional neural network

Semantic concept based video retrieval using convolutional neural network

Research on the Detection Method of Implicit Self Symmetry in a High-Level Semantic Model

A novel multilabel video retrieval method using multiple video queries and deep hash codes

Contact Info

Product

Resources

About