Continual spatio-temporal graph convolutional networks

Hedegaard, Lukas; Heidari, Negar; Iosifidis, Alexandros

doi:10.1016/j.patcog.2023.109528

Cited by 8 publications

(2 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…CoAGCN [ 52 ]: This method constructs an efficient skeleton-based online action recognition method by stepwise inputting continuous frame sequences into a graph convolutional network.…”

Section: Experimental Results and Analysismentioning

confidence: 99%

ST-TGR: Spatio-Temporal Representation Learning for Skeleton-Based Teaching Gesture Recognition

Chen,

Huang,

Liu

et al. 2024

Sensors

View full text Add to dashboard Cite

Teaching gesture recognition is a technique used to recognize the hand movements of teachers in classroom teaching scenarios. This technology is widely used in education, including for classroom teaching evaluation, enhancing online teaching, and assisting special education. However, current research on gesture recognition in teaching mainly focuses on detecting the static gestures of individual students and analyzing their classroom behavior. To analyze the teacher’s gestures and mitigate the difficulty of single-target dynamic gesture recognition in multi-person teaching scenarios, this paper proposes skeleton-based teaching gesture recognition (ST-TGR), which learns through spatio-temporal representation. This method mainly uses the human pose estimation technique RTMPose to extract the coordinates of the keypoints of the teacher’s skeleton and then inputs the recognized sequence of the teacher’s skeleton into the MoGRU action recognition network for classifying gesture actions. The MoGRU action recognition module mainly learns the spatio-temporal representation of target actions by stacking a multi-scale bidirectional gated recurrent unit (BiGRU) and using improved attention mechanism modules. To validate the generalization of the action recognition network model, we conducted comparative experiments on datasets including NTU RGB+D 60, UT-Kinect Action3D, SBU Kinect Interaction, and Florence 3D. The results indicate that, compared with most existing baseline models, the model proposed in this article exhibits better performance in recognition accuracy and speed.

show abstract

“…CoAGCN [ 52 ]: This method constructs an efficient skeleton-based online action recognition method by stepwise inputting continuous frame sequences into a graph convolutional network.…”

Section: Experimental Results and Analysismentioning

confidence: 99%

ST-TGR: Spatio-Temporal Representation Learning for Skeleton-Based Teaching Gesture Recognition

Chen,

Huang,

Liu

et al. 2024

Sensors

View full text Add to dashboard Cite

show abstract

“…When working with heterogeneous data, relative inductive bias can be introduced through guiding models in learning dependencies between sensors. Several works in HAR use wearable sensor data and skeleton data that exemplify this: variants of spatial–temporal graph convolution network [ 44 , 45 ] and variants of residual graph convolutional networks [ 46 , 47 ]. While graph convolution networks have been successful in HAR for wearables, we show in this work that they tend not to be as successful as attention-based graph neural networks in HAR for smart homes, especially since attention-based models are able to leverage the prior knowledge, indicating that some neighbors might be more informative than others.…”

Section: Introductionmentioning

confidence: 99%

Using Graphs to Perform Effective Sensor-Based Human Activity Recognition in Smart Homes

Plötz

2024

Sensors

View full text Add to dashboard Cite

There has been a resurgence of applications focused on human activity recognition (HAR) in smart homes, especially in the field of ambient intelligence and assisted-living technologies. However, such applications present numerous significant challenges to any automated analysis system operating in the real world, such as variability, sparsity, and noise in sensor measurements. Although state-of-the-art HAR systems have made considerable strides in addressing some of these challenges, they suffer from a practical limitation: they require successful pre-segmentation of continuous sensor data streams prior to automated recognition, i.e., they assume that an oracle is present during deployment, and that it is capable of identifying time windows of interest across discrete sensor events. To overcome this limitation, we propose a novel graph-guided neural network approach that performs activity recognition by learning explicit co-firing relationships between sensors. We accomplish this by learning a more expressive graph structure representing the sensor network in a smart home in a data-driven manner. Our approach maps discrete input sensor measurements to a feature space through the application of attention mechanisms and hierarchical pooling of node embeddings. We demonstrate the effectiveness of our proposed approach by conducting several experiments on CASAS datasets, showing that the resulting graph-guided neural network outperforms the state-of-the-art method for HAR in smart homes across multiple datasets and by large margins. These results are promising because they push HAR for smart homes closer to real-world applications.

show abstract

Continual Inference: A Library for Efficient Online Inference with Deep Neural Networks in PyTorch

Hedegaard

Iosifidis

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Continual spatio-temporal graph convolutional networks

Cited by 8 publications

References 11 publications

ST-TGR: Spatio-Temporal Representation Learning for Skeleton-Based Teaching Gesture Recognition

ST-TGR: Spatio-Temporal Representation Learning for Skeleton-Based Teaching Gesture Recognition

Using Graphs to Perform Effective Sensor-Based Human Activity Recognition in Smart Homes

Continual Inference: A Library for Efficient Online Inference with Deep Neural Networks in PyTorch

Contact Info

Product

Resources

About