TACNet: Transition-Aware Context Network for Spatio-Temporal Action Detection

Abstract

Figure 1: Diagram of transitional state. There are some ambiguous states around but not belong to the target actions, and it is hard to distinguish them. We define the states as "transitional state" (red boxes). If we can effectively distinguish these states, we can improve the ability of temporal extent detection. AbstractCurrent state-of-the-art approaches for spatio-temporal action detection have achieved impressive results but remain unsatisfactory for temporal extent detection. The main reason comes from that, there are some ambiguous states similar to the real actions which may be treated as target actions even by a well-trained network. In this paper, we define these ambiguous samples as "transitional states", and propose a Transition-Aware Context Network (TACNet) to distinguish transitional states. The proposed TACNet includes two main components, i.e., temporal context detector and transition-aware classifier. The temporal context detector can extract long-term context information with constant time complexity by constructing a recurrent network. The transition-aware classifier can further distinguish transitional states by classifying action and transitional states simultaneously. Therefore, the proposed TACNet can substantially improve the performance of spatio-temporal action detection. We extensively evaluate the proposed TAC-Net on UCF101-24 and J-HMDB datasets. The experimental results demonstrate that TACNet obtains competitive performance on JHMDB and significantly outperforms * indicates equal contribution.† indicates corresponding author.the state-of-the-art methods on the untrimmed UCF101-24 in terms of both frame-mAP and video-mAP.

Attention Multi-Scale Network for Automatic Layer Extraction of Ice Radar Topological Sequences

“…Their effect has been confirmed in other fields [25][26][27][28][29][30][31][32][33][34][35]39], such as object detection, anomaly detection and so on. Multi-scale modules [25][26][27][28][29] use different scale branches to collect complementary and different levels of ice sheet radar image features and merge them into multi-scale features to remedy the problem of poor feature extraction ability with a single-scale method. The attention modules [30][31][32][33][34] assign weights to different types of features from the global perspective to suppress noise, refine important ice boundary features and fit boundaries in radar topology sequences.…”

“…Therefore, to make up for the problems, the following two aspects are considered in our study. In order to further improve the representation of the features of ice layers adjacent regions, multi-scale features [25][26][27][28][29] can be used to extract more abundant scale features of ice layers. In addition, an attention mechanism can also be used to capture long-term relationships between ice layers and fuse the context information of radar images by paying attention to the features of the ice sheet radar images at different levels [30][31][32][33][34].…”

“…The approximate positions and the paths for collecting the dataset used in this paper are shown in Figure 1. There are 102 segments (a total of 5 profiles) containing basal terrain data of the Canadian Arctic Archipelago (CAA) (66 • N-83 • N, 61 • W-97 • W), each of which is about 50 km of 3D topological sequence [28] and can be used to reconstruct the 3D under-ice terrain of the target area. Three antenna subarrays, each of which contains 5 parallel antenna elements, are installed on the left wing, right wing and bottom of the fuselage.…”

A Survey on Temporal Action Localization

“…Compared the method two-stream and 3D convolution, two-stream is more accurate but less efficient than the latter. How to take the advantages of both of them better is a possible research direction in b) Action detection will extend from temporal action detection to spatio-temporal action detection [39]. That is to say, we should detect from one-dimensional temporal interval to two-dimensional spatio-temporal box that can detect actions more comprehensively.…”

Multimodal Spatiotemporal Networks for Sign Language Recognition

“…They utilize the Sobel operator and element-wise subtraction to calculate the spatial and temporal gradients respectively. Song et al [26] propose Discriminative Motion Cue (DMC) to reduce noises in motion vectors and capture fine motion details. They train the DMC generator to approximate flow using a reconstruction loss and an adversarial loss, jointly with the downstream action classification task.…”