IOD-CNN: Integrating object detection networks for event recognition

Abstract

Many previous methods have showed the importance of considering semantically relevant objects for performing event recognition, yet none of the methods have exploited the power of deep convolutional neural networks to directly integrate relevant object information into a unified network. We present a novel unified deep CNN architecture which integrates architecturally different, yet semantically-related object detection networks to enhance the performance of the event recognition task. Our architecture allows the sharing of the convolutional layers and a fully connected layer which effectively integrates event recognition, rigid object detection and non-rigid object detection.

Exploitation of Semantic Keywords for Malicious Event Classification

“…The next three rows show the classification accuracy based on a different baseline classifier ('Event CNN+'). This baseline also does not exploit any keyword information and is reported [5] to have used additional treatments such as an ROI pooling and a different training scheme. IOD-CNN [5] which embeds the keyword-driven object information by early-fusion outperforms its baseline ('Event CNN+') by 3.4 AP.…”

“…These "machine-driven" attention maps show that they clearly share high relevance with the "human-driven" semantic keywords although the classifier is not supported with any additional semantic information in the learning process. Lastly, we carry out a study to verify the practicality of explicitly incorporating these semantic keywords using various fusion approaches, which include a novel CNN-based architecture (IOD-CNN) developed by part of the authors [5]. We show that these keyword-driven information is effective in helping out the event classification task regardless of whether the information is used in an early-or late-fusion scheme.…”

Detecting Suspicious Behavior: How to Deal with Visual Similarity through Neural Networks

“…Convolutional Neural Networks (CNN) have a high performance in computer vision and pattern recognition. Many approaches implemented them to tackle problems such as object detection [29,30], identifying actions in images [31], and text recognition [32]. Ji et al [33] proposed a three-dimensional convolution on a CNN (3DCNN) architecture to analyze video data.…”

Is Pretraining Necessary for hyperspectral image classification?

“…Since the Pavia Center dataset has much more data than the others, it requires more iterations than the others. To cope with this issue, we adopt a two-step optimization strategy introduced in [7,8]. Under this scheme, the network is initially trained on only the largest dataset (Step I (PC)) and then is updated using the whole dataset for multi-task learning (Step II).…”