CatNet: Class Incremental 3D ConvNets for Lifelong Egocentric Gesture Recognition

Wang, Zhengwei; She, Qi; Chalasani, Tejo; Smolić, Aljoša

doi:10.1109/cvprw50498.2020.00123

Cited by 11 publications

(4 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We compare our method with the other state-of-the-art methods [38][39][40][41][42][43][44][45][46][47][48][49] on EgoGesture [12] SKIG [13] and IsoGD [14] datasets. It can be seen in Tables 4, 5, and 6.…”

Section: Final Results Of the Egogesture Skig And Isogd Datasetsmentioning

confidence: 99%

“…Table 4 Comparison results of our models with other state-of-the-art methods on the test set of EgoGesture [12] dataset Networks Accuracy (%) RGB Depth VGG16 [38] 62.50 62.30 C3D [7] 86.88 88.45 C3D + LSTM + RSTTM [39] 89.30 90.60 CatNet [40] 90.05 90.09 MTUT [41] 92.48 91.96 SeST [11] 93.20 93.35 ResNeXt-101 [42] 93.75 94.03 STCA-R(2 + 1)D [43] 94.00 -ACTION-Net [44] 94…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Parallel temporal feature selection based on improved attention mechanism for dynamic gesture recognition

Chen

Dong

Wang

et al. 2022

Complex Intell. Syst.

View full text Add to dashboard Cite

Dynamic gesture recognition has become a new type of interaction to meet the needs of daily interaction. It is the most natural, easy to operate, and intuitive, so it has a wide range of applications. The accuracy of gesture recognition depends on the ability to accurately learn the short-term and long-term spatiotemporal features of gestures. Our work is different from improving the performance of a single type of network with convnets-based models and recurrent neural network-based models or serial stacking of two heterogeneous networks, we proposed a fusion architecture that can simultaneously learn short-term and long-term spatiotemporal features of gestures, which combined convnets-based models and recurrent neural network-based models in parallel. At each stage of feature learning, the short-term and long-term spatiotemporal features of gestures are captured simultaneously, and the contribution of two heterogeneous networks to the classification results in spatial and channel axes that can be learned automatically by using the attention mechanism. The sequence and pooling operation of the channel attention module and spatial attention module are compared through experiments. And the proportion of short-term and long-term features of gestures on channel and spatial axes in each stage of feature learning is quantitatively analyzed, and the final model is determined according to the experimental results. The module can be used for end-to-end learning and the proposed method was validated on the EgoGesture, SKIG, and IsoGD datasets and got very competitive performance.

show abstract

Section: Final Results Of the Egogesture Skig And Isogd Datasetsmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Parallel temporal feature selection based on improved attention mechanism for dynamic gesture recognition

Chen

Dong

Wang

et al. 2022

Complex Intell. Syst.

View full text Add to dashboard Cite

show abstract

“…Although class-incremental learning has been studied for image classification, the research is also active for other applications, including person re-identification [42], 3D object classification [5], object detection [33], and semantic segmentation [24]. Continual learning in the video domain is rare [26,41]. Despite remarkable technical advances in action recognition, catastrophic forgetting problem has not been explored actively yet.…”

Section: Class-incremental Learning In Other Domainsmentioning

confidence: 99%

“…Despite remarkable technical advances in action recognition, catastrophic forgetting problem has not been explored actively yet. An existing approach [41] is limited to applying the iCaRL [30] based on a two-stream 3D convolutional neural network in a straightforward manner. On the other hand, our approach is based on knowledge distillation similar to [4,6] and exploits an attention method over a time-channel space intuitively to facilitate action recognition in a class-incremental learning scenario.…”

Section: Class-incremental Learning In Other Domainsmentioning

confidence: 99%

Class-Incremental Learning for Action Recognition in Videos

Park¹,

Kang²,

Han³

2022

Preprint

View full text Add to dashboard Cite

We tackle catastrophic forgetting problem in the context of class-incremental learning for video recognition, which has not been explored actively despite the popularity of continual learning. Our framework addresses this challenging task by introducing time-channel importance maps and exploiting the importance maps for learning the representations of incoming examples via knowledge distillation. We also incorporate a regularization scheme in our objective function, which encourages individual features obtained from different time steps in a video to be uncorrelated and eventually improves accuracy by alleviating catastrophic forgetting. We evaluate the proposed approach on brand-new splits of class-incremental action recognition benchmarks constructed upon the UCF101, HMDB51, and Something-Something V2 datasets, and demonstrate the effectiveness of our algorithm in comparison to the existing continual learning methods that are originally designed for image data.

show abstract