Exploring Complex Dependencies for Multi-modal Semantic Trajectory Prediction

Liu, Jie; Zhang, Lei; Zhu, Shaojie; Liu, Bailong; Liang, Zhizheng; Yang, Susong

doi:10.1007/s11063-021-10666-9

Cited by 3 publications

(2 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multi-modal feature fusion can be achieved by using a multi-modal fusion module (MFM) that integrates features from different aspects, yielding superior results compared to single modality. Despite that much progress has been reported on multi-modal feature learning [58][59][60], the proposed MFM module exhibits superior capability in extracting gait motion information, particularly when confronting with significant changes in walking conditions. • The proposed object region extraction algorithm, namely Gait-YOLO, partially addresses the challenge of varying target receptive fields.…”

Section: Discussionmentioning

confidence: 99%

Human Gait Recognition Based on Frontal-View Walking Sequences Using Multi-modal Feature Representations and Learning

Deng,

Zhong,

Zou

et al. 2024

Neural Process Lett

View full text Add to dashboard Cite

Despite that much progress has been reported in gait recognition, most of these existing works adopt lateral-view parameters as gait features, which requires large area of data collection environment and limits the applications of gait recognition in real-world practice. In this paper, we adopt frontal-view walking sequences rather than lateral-view sequences and propose a new gait recognition method based on multi-modal feature representations and learning. Specifically, we characterize walking sequences with two different kinds of frontal-view gait features representations, including holistic silhouette and dense optical flow. Pedestrian regions extraction is achieved by an improved YOLOv7 algorithm called Gait-YOLO algorithm to eliminate the effects of background interference. Multi-modal fusion module (MFM) is proposed to explore the intrinsic connections between silhouette and dense optical flow features by using squeeze and excitation operations at the channel and spatial levels. Gait feature encoder is further used to extract global walking characteristics, enabling efficient multi-modal information fusion. To validate the efficacy of the proposed method, we conduct experiments on CASIA-B and OUMVLP gait databases and compare performance of our proposed method with other existing state-of-the-art gait recognition methods.

show abstract

Section: Discussionmentioning

confidence: 99%

Human Gait Recognition Based on Frontal-View Walking Sequences Using Multi-modal Feature Representations and Learning

Deng,

Zhong,

Zou

et al. 2024

Neural Process Lett

View full text Add to dashboard Cite

show abstract

“…Considering the important role of the semantic layer for target behaviour analysis, scholars have semantically enriched trajectory [6][7][8] and explored semantic representation methods [9][10][11]. On this basis, some scholars performed the cluster analysis [12][13][14] of trajectory points by clustering around discrete semantic information, such as geographic tags [15] and attribute tags [16].…”

Section: Related Workmentioning

confidence: 99%

A trajectory summarisation generation method based on the mobile robot behaviour analysis

Liu

et al. 2022

IET Cyber-Syst and Robotics

View full text Add to dashboard Cite

The semantic representation of the trajectory is conducive to enrich the content of trajectory data mining. A trajectory summarisation generation method based on the mobile robot behaviour analysis was proposed to realize the abstract expression and semantic representation of the spatio-temporal motion features of the robot and its environmental interaction state. First, the behavioural semantic modelling and representation of the mobile robot are completed by modelling the sub-trajectory and calculating the topological behaviour (TOP). Second, Chinese word segmentation and semantic slot filling methods are used to combine with hierarchical clustering to perform basic word extraction and classification for describing trajectory sentences. Then, the description language frame is extracted based on the TOP, and the final trajectory summarisation is generated. The result shows that the proposed method can semantically represent robot behaviours with different motion features and topological features, extract two verb-frameworks for describing the sentences according to their topological features, and dynamically adjust the syntactic structure for the different topological behaviours between the target and the environment. The proposed method can generate semantic information of relatively high quality for spatio-temporal data and help to understand the higher-order semantics of moving robot behaviour.

show abstract

Hybrid artificial bee colony based neural network and dynamic threshold technique for predicting moving vehicle location and co-located objects

Anitha¹,

Sivaprakash

Velmurugan³

et al. 2023

Sādhanā

View full text Add to dashboard Cite

Exploring Complex Dependencies for Multi-modal Semantic Trajectory Prediction

Cited by 3 publications

References 37 publications

Human Gait Recognition Based on Frontal-View Walking Sequences Using Multi-modal Feature Representations and Learning

Human Gait Recognition Based on Frontal-View Walking Sequences Using Multi-modal Feature Representations and Learning

A trajectory summarisation generation method based on the mobile robot behaviour analysis

Hybrid artificial bee colony based neural network and dynamic threshold technique for predicting moving vehicle location and co-located objects

Contact Info

Product

Resources

About