Complex Events Detection Using Data-Driven Concepts

Yang, Yang; Shah, Mubarak

doi:10.1007/978-3-642-33712-3_52

Cited by 50 publications

(46 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Liu et al [16] combine a set of manually specified attributes with data-driven attributes for action recognition. Yang and Shah [27] propose the use of data-driven concepts for event detection in video. Mensink et al [17] propose an efficient method for generalizing to new categories in image classification by extending the nearest class mean (NCM) classifier that is reminiscent of category-level features.…”

Section: Data-driven Category-level Featuresmentioning

confidence: 99%

DaMN – Discriminative and Mutually Nearest: Exploiting Pairwise Category Proximity for Video Action Recognition

Hou¹,

Zamir²,

Sukthankar

et al. 2014

Computer Vision – ECCV 2014

Self Cite

View full text Add to dashboard Cite

Abstract. We propose a method for learning discriminative categorylevel features and demonstrate state-of-the-art results on large-scale action recognition in video. The key observation is that one-vs-rest classifiers, which are ubiquitously employed for this task, face challenges in separating very similar categories (such as running vs. jogging). Our proposed method automatically identifies such pairs of categories using a criterion of mutual pairwise proximity in the (kernelized) feature space, using a category-level similarity matrix where each entry corresponds to the one-vs-one SVM margin for pairs of categories. We then exploit the observation that while splitting such "Siamese Twin" categories may be difficult, separating them from the remaining categories in a two-vs-rest framework is not. This enables us to augment one-vs-rest classifiers with a judicious selection of "two-vs-rest" classifier outputs, formed from such discriminative and mutually nearest (DaMN) pairs. By combining onevs-rest and two-vs-rest features in a principled probabilistic manner, we achieve state-of-the-art results on the UCF101 and HMDB51 datasets. More importantly, the same DaMN features, when treated as a mid-level representation also outperform existing methods in knowledge transfer experiments, both cross-dataset from UCF101 to HMDB51 and to new categories with limited training data (one-shot and few-shot learning). Finally, we study the generality of the proposed approach by applying DaMN to other classification tasks; our experiments show that DaMN outperforms related approaches in direct comparisons, not only on video action recognition but also on their original image dataset tasks.

show abstract

Section: Data-driven Category-level Featuresmentioning

confidence: 99%

DaMN – Discriminative and Mutually Nearest: Exploiting Pairwise Category Proximity for Video Action Recognition

Hou¹,

Zamir²,

Sukthankar

et al. 2014

Computer Vision – ECCV 2014

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, semantic concepts have the benefit of supervised learning since they have labels. To alleviate the issue of manual annotation, there are quite a few recent works that discover 'concepts' automatically and use them to improve detection [26], classification [18], recognition [13,27] and retrieval [28]. These concepts are learned automatically in an unsupervised fashion from the data.…”

Section: Introductionmentioning

confidence: 99%

Improving Semantic Concept Detection through the Dictionary of Visually-Distinct Elements

Dehghan

Idrees

Shah

2014

2014 IEEE Conference on Computer Vision and Pattern Recognition

Self Cite

View full text Add to dashboard Cite

A video captures a sequence and interactions of concepts that can be static, for instance, objects or scenes, or dynamic, such as actions. For large datasets containing hundreds of thousands of images or videos, it is impractical to manually annotate all the concepts, or all the instances of a single concept. However, a dictionary with visuallydistinct elements can be created automatically from unlabeled videos which can capture and express the entire dataset. The downside to this machine-discovered dictionary is meaninglessness, i.e., its elements are devoid of semantics and interpretation. In this paper, we present an approach that leverages the strengths of semantic concepts and the machine-discovered DOVE by learning a relationship between them. Since instances of a semantic concept share visual similarity, the proposed approach uses softconsensus regularization to learn the mapping that enforces instances from each semantic concept to have similar representations. The testing is performed by projecting the query onto the DOVE as well as new representations of semantic concepts from training, with non-negativity and unit summation constraints for probabilistic interpretation. We tested our formulation on TRECVID MED and SIN tasks, and obtained encouraging results.

show abstract

“…This result is very important as it shows that, despite their complex nature, many events can be well characterized by features of low-level semantics [6,7,8]. However, hierarchical approaches have also become increasingly popular in recent years where more general "concepts" are first identified and then used as atoms for the characterization and recognition of complex events [9,10,6,7].…”

Section: Introduction and Related Workmentioning

confidence: 99%

“…Concepts have also been deployed in the form of attributes [11], which can be considered as concepts with small granularity [6]. [7] used deep learning to find data-driven concepts. Data-driven concepts are an interesting idea and have shown promising performance: however, they are harder to link to a conceptual description of the videos.…”

Section: Introduction and Related Workmentioning

confidence: 99%

Complex event recognition by latent temporal models of concepts

Borzeshi

Dehghan

Piccardi

et al. 2014

2014 IEEE International Conference on Image Processing (ICIP)

Self Cite

View full text Add to dashboard Cite

Complex event recognition is an expanding research area aiming to recognize entities of high-level semantics in videos. Typical approaches exploit the so-called "bags" of spatiotemporal features such as STIP, ISA and DTF-HOG; yet, more recently, the notion of concept has emerged as an alternative, intermediate representation with greater descriptive power, and "bags of concepts" have been used for recognition. In this paper we argue that concepts in an event tend to articulate over a discernible temporal structure and we exploit a temporal model using the scores of concept detectors as measurements. In addition, we propose several heuristics to improve the initialization of the model's latent states and take advantage of the time-sparsity of the concepts. Experimental results on videos from the challenging TRECVID MED 2012 dataset show that the proposed approach achieves an improvement in average precision of 8.92% over comparable bags of concepts, thus validating the use of temporal structure over concepts for complex event recognition.

show abstract

Complex Events Detection Using Data-Driven Concepts

Cited by 50 publications

References 22 publications

DaMN – Discriminative and Mutually Nearest: Exploiting Pairwise Category Proximity for Video Action Recognition

DaMN – Discriminative and Mutually Nearest: Exploiting Pairwise Category Proximity for Video Action Recognition

Improving Semantic Concept Detection through the Dictionary of Visually-Distinct Elements

Complex event recognition by latent temporal models of concepts

Contact Info

Product

Resources

About