InfAR dataset: Infrared action recognition at different times

Gao, Chenqiang; Du, Yinhe; Liu, Jiang; Lv, Jing; Yang, Lu; Meng, Deyu; Hauptmann, Alexander G.

doi:10.1016/j.neucom.2016.05.094

Cited by 72 publications

(53 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The previous work [3] shows that the temporal information outperforms the appearance information in the infrared action recognition. However, to our best knowledge, there is no work that uses the whole cycle of an action for infrared action recognition.…”

Section: A Global Temporal Stream Based Cnns Configurationsmentioning

confidence: 99%

“…Initially, we evaluate the performance of each stream of the three-stream CNNs, spatial stream with raw infrared images as input, some of their combinations, and our proposed TSTDDs on InfAR dataset, shown in Table I. Then, the performance of TSTDDs is compared with that of representative state-of-the-art handcrafted features based methods improved dense trajectories (iDTs) [26], Dense-Traj [29], HOF [3] and deep learning features based methods TDDs [7], Two-stream CNNs [8], and 3-D CNNs [4]. All the comparison results are summarized in Table II.…”

Section: Evaluations On Infar and Ntu Rgb+d Datasetsmentioning

confidence: 99%

“…Recently, some deep learning based methods have been proposed for infrared action recognition. Gao et al [3] adopted a two-stream CNN architecture to evaluate their newly built infrared action recognition dataset InfAR. This architecture consists of an optical-flow motion-history-image (OF-MHI) stream network and an OF stream network.…”

Section: Introductionmentioning

confidence: 99%

“…This architecture contains two separated recognition networks: Original infrared image net and OF net. They achieved better performance than that in [3] by introducing the 3-D CNN architecture and the discriminative code loss of the objective function. In these methods, the temporal features are local and generally consider only two (or a few) consecutive images in the video sequence.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Global Temporal Representation Based CNNs for Infrared Action Recognition

Liu

et al. 2018

IEEE Signal Process. Lett.

View full text Add to dashboard Cite

Infrared human action recognition has many advantages, i.e., it is insensitive to illumination change, appearance variability, and shadows. Existing methods for infrared action recognition are either based on spatial or local temporal information, however, the global temporal information, which can better describe the movements of body parts across the whole video, is not considered. In this letter, we propose a novel global temporal representation named optical-flow stacked difference image (OFSDI) and extract robust and discriminative feature from the infrared action data by considering the local, global, and spatial temporal information together. Due to the small size of the infrared action dataset, we first apply convolutional neural networks on local, spatial, and global temporal stream respectively to obtain efficient convolutional feature maps from the raw data rather than train a classifier directly. Then these convolutional feature maps are aggregated into effective descriptors named three-stream trajectory-pooled deep-convolutional descriptors by trajectory-constrained pooling. Furthermore, we improve the robustness of these features by using the locality-constrained linear coding (LLC) method. With these features, a linear support vector machine (SVM) is adopted to classify the action data in our scheme. We conduct the experiments on infrared action recognition datasets InfAR and NTU RGB+D. The experimental results show that the proposed approach outperforms the representative state-of-the-art handcrafted features and deep learning features based methods for the infrared action recognition. Index Terms-Convolutional neural networks (CNN), deep learning, global temporal information, infrared action recognition.1070-9908

show abstract

Section: A Global Temporal Stream Based Cnns Configurationsmentioning

confidence: 99%

Section: Evaluations On Infar and Ntu Rgb+d Datasetsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Global Temporal Representation Based CNNs for Infrared Action Recognition

Liu

et al. 2018

IEEE Signal Process. Lett.

View full text Add to dashboard Cite

show abstract

“…However, the methods for infrared action recognition are limited. Furthermore, there is only one publicly available infrared dataset InfAR [20] for action recognition until now. As a result, the performance of infrared action recognition in previous works is preliminary and leaves a reasonable space to further promote its performance.…”

Section: Complexitymentioning

confidence: 99%

Transferable Feature Representation for Visible‐to‐Infrared Cross‐Dataset Human Action Recognition

Liu

et al. 2018

Complexity

View full text Add to dashboard Cite

Recently, infrared human action recognition has attracted increasing attention for it has many advantages over visible light, that is, being robust to illumination change and shadows. However, the infrared action data is limited until now, which degrades the performance of infrared action recognition. Motivated by the idea of transfer learning, an infrared human action recognition framework using auxiliary data from visible light is proposed to solve the problem of limited infrared action data. In the proposed framework, we first construct a novel Cross-Dataset Feature Alignment and Generalization (CDFAG) framework to map the infrared data and visible light data into a common feature space, where Kernel Manifold Alignment (KEMA) and a dual alignedto-generalized encoders (AGE) model are employed to represent the feature. Then, a support vector machine (SVM) is trained, using both the infrared data and visible light data, and can classify the features derived from infrared data. The proposed method is evaluated on InfAR, which is a publicly available infrared human action dataset. To build up auxiliary data, we set up a novel visible light action dataset XD145. Experimental results show that the proposed method can achieve state-of-the-art performance compared with several transfer learning and domain adaptation methods.

show abstract

PM-GANs: Discriminative Representation Learning for Action Recognition Using Partial-Modalities

Wang

Gao

Yang

et al. 2018

Computer Vision – ECCV 2018

Self Cite

View full text Add to dashboard Cite

Data of different modalities generally convey complimentary but heterogeneous information, and a more discriminative representation is often preferred by combining multiple data modalities like the RGB and infrared features. However in reality, obtaining both data channels is challenging due to many limitations. For example, the RGB surveillance cameras are often restricted from private spaces, which is in conflict with the need of abnormal activity detection for personal security. As a result, using partial data channels to build a full representation of multi-modalities is clearly desired. In this paper, we propose a novel Partial-modal Generative Adversarial Networks (PM-GANs) that learns a full-modal representation using data from only partial modalities. The full representation is achieved by a generated representation in place of the missing data channel. Extensive experiments are conducted to verify the performance of our proposed method on action recognition, compared with four state-of-the-art methods. Meanwhile, a new Infrared-Visible Dataset for action recognition is introduced, and will be the first publicly available action dataset that contains paired infrared and visible spectrum.

show abstract

InfAR dataset: Infrared action recognition at different times

Cited by 72 publications

References 47 publications

Global Temporal Representation Based CNNs for Infrared Action Recognition

Global Temporal Representation Based CNNs for Infrared Action Recognition

Transferable Feature Representation for Visible‐to‐Infrared Cross‐Dataset Human Action Recognition

PM-GANs: Discriminative Representation Learning for Action Recognition Using Partial-Modalities

Contact Info

Product

Resources

About