OFPI: Optical Flow Pose Image for Action Recognition

Chen, Dong; Tao, Zhang; Zhou, Peng; Yan, Chenyang; Li, Chuanqi

doi:10.3390/math11061451

Cited by 3 publications

(2 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The models were independently trained and evaluated, allowing us to provide a comprehensive comparison of performance across various deep learning architectures. Consequently, they serve as a holistic platform for assessing the effectiveness of our proposed method [16,[31][32][33][34][35].…”

Section: Network Architecture and Training Processmentioning

confidence: 99%

Enhancing Robustness of Viewpoint Changes in 3D Skeleton-Based Human Action Recognition

Park

Kim

2023

Mathematics

View full text Add to dashboard Cite

Previous research on 3D skeleton-based human action recognition has frequently relied on a sequence-wise viewpoint normalization process, which adjusts the view directions of all segmented action sequences. This type of approach typically demonstrates robustness against variations in viewpoint found in short-term videos, a characteristic commonly encountered in public datasets. However, our preliminary investigation of complex action sequences, such as discussions or smoking, reveals its limitations in capturing the intricacies of such actions. To address these view-dependency issues, we propose a straightforward, yet effective, sequence-wise augmentation technique. This strategy enhances the robustness of action recognition models, particularly against changes in viewing direction that mainly occur within the horizontal plane (azimuth) by rotating human key points around either the z-axis or the spine vector, effectively creating variations in viewing directions. We scrutinize the robustness of this approach against real-world viewpoint variations through extensive empirical studies on multiple public datasets, including an additional set of custom action sequences. Despite the simplicity of our approach, our experimental results consistently yield improved action recognition accuracies. Compared to the sequence-wise viewpoint normalization method used with advanced deep learning models like Conv1D, LSTM, and Transformer, our approach showed a relative increase in accuracy of 34.42% for the z-axis and 10.86% for the spine vector.

show abstract

Section: Network Architecture and Training Processmentioning

confidence: 99%

Enhancing Robustness of Viewpoint Changes in 3D Skeleton-Based Human Action Recognition

Park

Kim

2023

Mathematics

View full text Add to dashboard Cite

show abstract

“…To address the issue of weak self-information and mutual information in feature matching, existing methods generally employ feature enhancement techniques for insufficient self-information. For example, a second-order similarity network (SoSN) [17] uses second-order statistics for feature strengthening; alternatively, optical flow [18][19][20] or skeletal motion [21,22] are added. To tackle the problem of insufficient mutual information, temporal alignment strategies [20,23,24] are typically employed.…”

Section: Introductionmentioning

confidence: 99%

HiTIM: Hierarchical Task Information Mining for Few-Shot Action Recognition

Jiang

Dang

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

Although the existing few-shot action recognition methods have achieved impressive results, they suffer from two major shortcomings. (a) During feature extraction, few-shot tasks are not distinguished and task-irrelevant features are obtained, resulting in the loss of task-specific critical discriminative information. (b) During feature matching, information critical to the features within the task, i.e., self-information and mutual information, is ignored, resulting in the accuracy being affected by redundant or irrelevant information. To overcome these two limitations, we propose a hierarchical task information mining (HiTIM) approach for few-shot action recognition that incorporates two key components: an inter-task learner (Kinter) and an attention-matching module with an intra-task learner (Kintra). The purpose of the Kinter is to learn the knowledge of different tasks and build a task-related feature space for obtaining task-specific features. The proposed matching module with Kintra consists of two branches: the spatiotemporal self-attention matching (STM) and correlated cross-attention matching (CM), which reinforce key spatiotemporal information in features and mine regions with strong correlations between features, respectively. The shared Kintra can further optimize STM and CM. In our method, we can use either a 2D convolutional neural network (CNN) or 3D CNN as embedding. In comparable experiments using two different embeddings in the five-way one-shot and five-way five-shot task, the proposed method achieved recognition accuracy that outperformed other state-of-the-art (SOTA) few-shot action recognition methods on the HMDB51 dataset and was comparable to SOTA few-shot action recognition methods on the UCF101 and Kinetics datasets.

show abstract

Оценка Методов Скелетизации Двумерных Бинарных Изображений

Abudalfa

2023

View full text Add to dashboard Cite

In the realm of modern image processing, the emphasis often lies on engineering-based approaches rather than scientific solutions to address diverse practical problems. One prevalent task within this domain involves the skeletonization of binary images. Skeletonization is a powerful process for extracting the skeleton of objects located in digital binary images. This process is widely employed for automating many tasks in numerous fields such as pattern recognition, robot vision, animation, and image analysis. The existing skeletonization techniques are mainly based on three approaches: boundary erosion, distance coding, and Voronoi diagram for identifying an approximate skeleton. In this work, we present an empirical evaluation of a set of well-known techniques and report our findings. We specifically deal with computing skeletons in 2d binary images by selecting different approaches and evaluating their effectiveness. Visual evaluation is the primary method used to showcase the performance of selected skeletonization algorithms. Due to the absence of a definitive definition for the "true" skeleton of a digital object, accurately assessing the effectiveness of skeletonization algorithms poses a significant research challenge. Although researchers have attempted quantitative assessments, these measures are typically customized for specific domains and may not be suitable for our current work. The experimental results shown in this work illustrate the performance of the three main approaches in applying skeletonization with respect to different perspectives.

show abstract

OFPI: Optical Flow Pose Image for Action Recognition

Cited by 3 publications

References 31 publications

Enhancing Robustness of Viewpoint Changes in 3D Skeleton-Based Human Action Recognition

Enhancing Robustness of Viewpoint Changes in 3D Skeleton-Based Human Action Recognition

HiTIM: Hierarchical Task Information Mining for Few-Shot Action Recognition

Оценка Методов Скелетизации Двумерных Бинарных Изображений

Contact Info

Product

Resources

About