Sample-Specific Late Fusion for Visual Category Recognition

Dong, Liyun; Lai, Kevin; Ye, Guangnan; Chen, Ming-Syan; Chang, Shih-Fu

doi:10.1109/cvpr.2013.109

Cited by 63 publications

(32 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…It seeks a shared low-rank matrix to remove noises of certain modalities, which requires to iteratively compute singular value decomposition with a cubic time complexity, and thus is less scalable for large scale real-world applications. In a following up work by Liu et al [25], dynamic fusion was adopted to find the best feature combination for each sample. This approach was proved effective but is extremely time-consuming.…”

Section: Fusing Multiple Featuresmentioning

confidence: 99%

Exploring Inter-feature and Inter-class Relationships with Deep Neural Networks for Video Classification

Jiang

Wang

et al. 2014

Proceedings of the 22nd ACM International Conference on Multimedia

128

View full text Add to dashboard Cite

Videos contain very rich semantics and are intrinsically multimodal. In this paper, we study the challenging task of classifying videos according to their high-level semantics such as human actions or complex events. Although extensive efforts have been paid to study this problem, most existing works combined multiple features using simple fusion strategies and neglected the exploration of inter-class semantic relationships. In this paper, we propose a novel unified framework that jointly learns feature relationships and exploits the class relationships for improved video classification performance. Specifically, these two types of relationships are learned and utilized by rigorously imposing regularizations in a deep neural network (DNN). Such a regularized DNN can be efficiently launched using a GPU implementation with an affordable training cost. Through arming the DNN with better capability of exploring both the interfeature and the inter-class relationships, the proposed regularized DNN is more suitable for identifying video semantics. With extensive experimental evaluations, we demonstrate that the proposed framework exhibits superior performance over several state-of-the-art approaches. On the well-known Hollywood2 and Columbia Consumer Video benchmarks, we obtain to-date the best reported results: 65.7% and 70.6% respectively in terms of mean average precision.

show abstract

Section: Fusing Multiple Featuresmentioning

confidence: 99%

Exploring Inter-feature and Inter-class Relationships with Deep Neural Networks for Video Classification

Jiang

Wang

et al. 2014

Proceedings of the 22nd ACM International Conference on Multimedia

128

View full text Add to dashboard Cite

show abstract

“…The straightforward way to multiple modalities is line weighted fusion that has been adopted in [8]. D. Liu et al [23] proposed Sample Specific Late Fusion (SSLF) method, which aims to learn an optimal samplespecific fusion weights and enforces positive samples have the highest fusion scores.…”

Section: B Multimodal Fusionmentioning

confidence: 99%

Exploring multimodal video representation for action recognition

Wang

Yang

Meinel

2016

2016 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

A video contains rich perceptual information, such as visual appearance, motion and audio, which can be used for understanding the activities in videos. Recent works have shown the combination of appearance (spatial) and motion (temporal) clues can significantly improve human action recognition performance in videos. To further explore the multimodal representation of video in action recognition, We propose a framework to learn a multimodal representations from video appearance, motion as well as audio data. Convolutional Neural Networks (CNN) are trained for each modality respectively. For fusing multiple features extracted with CNNs, we propose to add a fusion layer on the top of CNNs to learn a joint video representation. In fusion phase, we investigate both early fusion and late fusion with Neural Network and Support Vector Machine. Compare to existing works, (1) our work measures the benefits of taking audio information into consideration and (2) implements sophisticated fusion methods. The effectiveness of proposed approach is evaluated on UCF101 and UCF101-50 (selected subset in which each video contains audio data) for action recognition. The experimental results show that different modalities are complementary to each other and multimodal representation can be beneficial for final prediction. Furthermore, proposed fusion approach achieves 85.1% accuracy in fusing spatial-temporal on UCF101 (split 1), which is very competitive to state-of-the-art works.

show abstract

“…Traditional classifier-based approaches include linear classifiers with minimised least squares error [4] or L 1 -norm soft margin error [5,6], reduced multivariate polynomial classifier [7], support vector machine [8] and single hidden layer feedforward neural network [9]. Other advanced techniques in the pattern recognition society, such as semisupervised learning [10,11], ensemble learning [12] and kernel tricks [6], can also be transferred without difficulty to solve the fusion problem. Two recent examples of classifier-based algorithms are FWOT [13] and minCq [14,15].…”

Section: Introductionmentioning

confidence: 99%

Combining multiple biometric traits with an order-preserving score fusion algorithm

et al. 2016

View full text Add to dashboard Cite

Cite this article as: Yicong Liang, Xiaoqing Ding, Changsong Liu, Jing-Hao Xue, Combining multiple biometric traits with an order-preserving score fusion algorithm, Neurocomputing, http://dx. AbstractMultibiometric systems based on score fusion can effectively combine the discriminative power of multiple biometric traits and overcome the limitations of individual trait, leading to a better performance of biometric authentication. To tackle multiple adverse issues with the established classifierbased or probability-based algorithms, in this paper we propose a novel order-preserving probabilistic score fusion algorithm, Order-Preserving Tree (OPT), by casting the score fusion problem into an optimisation problem with the natural order-preserving constraint. OPT is an algorithm fully non-parametric and widely applicable, not assuming any parametric forms of probabilities or independence among sources, directly estimating the posterior probabilities from maximum likelihood estimation, and exploiting the power of tree-structured ensembles. We demonstrate the effectiveness of our OPT algorithm by comparing it with many widely-used score fusion algorithms on two prevalent multibiometric databases.

show abstract

Sample-Specific Late Fusion for Visual Category Recognition

Cited by 63 publications

References 19 publications

Exploring Inter-feature and Inter-class Relationships with Deep Neural Networks for Video Classification

Exploring Inter-feature and Inter-class Relationships with Deep Neural Networks for Video Classification

Exploring multimodal video representation for action recognition

Combining multiple biometric traits with an order-preserving score fusion algorithm

Contact Info

Product

Resources

About