Dronecaps: Recognition Of Human Actions In Drone Videos Using Capsule Networks With Binary Volume Comparisons

Abstract

Please refer to published version for the most recent bibliographic citation information. If a published version is known of, the repository item page linked to above, will contain details on accessing it.

DroneAttention: Sparse weighted temporal attention for drone-camera based activity recognition

“…Our approach outperforms the previously existing methods. It successfully achieves state-ofthe-art results in all three datasets, namely 72.76% on the Okutama dataset, 92.56% on the MOD20 RoiAlign leads the network to focus on the key points where the action is currently taking place, and ignores the background noise, as opposed to the previously used 3D CNNs in [58]. It is also computationally less expensive, being a single stream network as compared to the approaches used in the previous works for MOD20 and Okutama dataset evaluation.…”

“…The state-of-the-art on MOD20 dataset [3] uses a two-stream approach and depends on motion-CNN for its accuracy. The state-of-the-art on Okutama dataset [58] uses features computed by 3D convolution neural networks plus a new set of features computed by a binary volume comparison (BVC) layer. BVC layer comprises three parts: a 3D-Conv layer with 12 non-trainable (i.e., fixed) filters, a non-linear function, and a set of learnable weights.…”

SWTA: Sparse Weighted Temporal Attention for Drone-Based Activity Recognition

“…It successfully achieves state-of-the-art results in all three datasets, namely 72.76% on the Okutama dataset, 92.56% on the MOD20 dataset, and 71.79% on the Drone Action dataset without pose-stream whereas 78.86% with pose-stream. For the Okutama dataset specifically, our Weighted Temporal Attention module with RoiAlign leads the network to focus on the keypoints where the action is currently taking place, and ignores the background noise, as opposed to the previously used 3D CNNs in [33]. It is also computationally less expensive, being a single stream network as compared to the approaches used in the previous works for MOD20 and Okutama dataset evaluation.…”

“…The state-of-theart on MOD20 dataset [3] uses a two-stream approach and depends on motion-CNN for their accuracy. The state-of-theart on Okutama-Action dataset [33] uses features computed by 3D convolution neural networks plus a new set of features computed by Binary Volume Comparison (BVC) layer, which comprises three parts: a 3D-Conv layer with 12 non-trainable (i.e., fixed) filters, a non-linear function and a set of learnable weights. Features from both the streams: 3D CNNs and BVC layer are concatenated and passed to Capsule Network for final activity prediction.…”

Capsule Networks with Routing Annealing

“…Ever since, different routing algorithms and architectures for capsule networks have been proposed and have found applications in various tasks [2,4,11,18]. We refer to routing-based CapsNets as those models that employ a routing algorithm in the architecture of the network.…”