Spatio-Temporal Channel Correlation Networks for Action Classification

Diba, Ali; Fayyaz, Mohsen; Sharma, V.; Yousefzadeh, Rahman; Gall, Jüergen; Gool, Luc Van

doi:10.48550/arxiv.1806.07754

Cited by 2 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section, we study the proposed automatic method of designing action recognition network to demonstrate its advantages over other famous action recognition architectures, e.g., 3D-ResNet [19], C3D network [20], and STC-ResNet [21]. We evaluate our algorithm on the challenging action recognition dataset UCF101, which is a trimmed dataset containing 13320 video clips of 101 classes, with the training from scratch protocol.…”

Section: Methodsmentioning

confidence: 99%

Video Action Recognition Via Neural Architecture Searching

Peng

Hong

Zhao

2019

2019 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

Deep neural networks have achieved great success for video analysis and understanding. However, designing a highperformance neural architecture requires substantial efforts and expertise. In this paper, we make the first attempt to let algorithm automatically design neural networks for video action recognition tasks. Specifically, a spatio-temporal network is developed in a differentiable space modeled by a directed acyclic graph, thus a gradient-based strategy can be performed to search an optimal architecture. Nonetheless, it is computationally expensive, since the computational burden to evaluate each architecture candidate is still heavy. To alleviate this issue, we, for the video input, introduce a temporal segment approach to reduce the computational cost without losing global video information. For the architecture, we explore in an efficient search space by introducing pseudo 3D operators. Experiments show that, our architecture outperforms popular neural architectures, under the training from scratch protocol, on the challenging UCF101 dataset, surprisingly, with only around one percentage of parameters of its manual-design counterparts.

show abstract

Section: Methodsmentioning

confidence: 99%

Video Action Recognition Via Neural Architecture Searching

Peng

Hong

Zhao

2019

2019 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

show abstract

“…The notion of correlation in [7] refers to the relationship among the spatial and temporal dimensions of the feature maps, which is different from the matching of adjacent frames studied in our work. We compare our results with [7] in Sec. 6.3.…”

Section: Related Workmentioning

confidence: 95%

“…In the context of action recognition, Zhao et al [55] utilize the correlation layer to compute a cost volume to estimate the displacement map as in optical flow. The Spatio-Temporal Channel Correlation Network [7] adapts the Squeeze-and-Excitation block [18] to a ResNeXt [51] backbone. The notion of correlation in [7] refers to the relationship among the spatial and temporal dimensions of the feature maps, which is different from the matching of adjacent frames studied in our work.…”

Section: Related Workmentioning

confidence: 99%

“…The Spatio-Temporal Channel Correlation Network [7] adapts the Squeeze-and-Excitation block [18] to a ResNeXt [51] backbone. The notion of correlation in [7] refers to the relationship among the spatial and temporal dimensions of the feature maps, which is different from the matching of adjacent frames studied in our work. We compare our results with [7] in Sec.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Video Modeling With Correlation Networks

Wang

Tran

Torresani

et al. 2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

131

View full text Add to dashboard Cite

Motion is a salient cue to recognize actions in video. Modern action recognition models leverage motion information either explicitly by using optical flow as input or implicitly by means of 3D convolutional filters that simultaneously capture appearance and motion information. This paper proposes an alternative approach based on a learnable correlation operator that can be used to establish frameto-frame matches over convolutional feature maps in the different layers of the network. The proposed architecture enables the fusion of this explicit temporal matching information with traditional appearance cues captured by 2D convolution. Our correlation network compares favorably with widely-used 3D CNNs for video modeling, and achieves competitive results over the prominent two-stream network while being much faster to train. We empirically demonstrate that correlation networks produce strong results on a variety of video datasets, and outperform the state of the art on three popular benchmarks for action recognition: Kinetics, Something-Something and Diving48.

show abstract

Spatio-Temporal Channel Correlation Networks for Action Classification

Cited by 2 publications

References 0 publications

Video Action Recognition Via Neural Architecture Searching

Video Action Recognition Via Neural Architecture Searching

Video Modeling With Correlation Networks

Contact Info

Product

Resources

About