An Application of Convolutional Neural Networks on Human Intention Prediction

Zhang, Lin; Li, Shengchao; Xiong, Hao; Diao, Xiumin; Ma, Ou

doi:10.5121/ijaia.2019.10501

Cited by 3 publications

(2 citation statements)

References 20 publications

(3 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It automates feature engineering to obtain more robust features by learning-based way in an endto-end manner. In recent years, with the rise of artificial intelligence [19]- [21] and human-computer interaction technology [22]- [24], object detection and Gesture recognition methods based on deep learning have made innovative breakthroughs [25]- [27], e.g. RCNN [28], Fast R-CNN [29], SSD [9], YOLO [30], etc.…”

Section: B Object Detection Methods Based On Deep Learning and Lightmentioning

confidence: 99%

Hierarchical Attention-Based Astronaut Gesture Recognition: A Dataset and CNN Model

Zhang

Wang

2020

IEEE Access

View full text Add to dashboard Cite

In the space station, visual gesture recognition is an important component for gesture based human-robot interaction to control the robot, which assists astronauts to complete simple, repetitive and cooperative work. However, public gesture datasets are captured in the daily environment and existing approaches trained on these datasets achieve imprecise performance for astronaut gesture recognition. In this paper, we introduce a new astronaut gesture dataset (DSSL-Astronaut gesture dataset) and a novel hierarchical attention single-shot detector network (HA-SSD) for astronaut gesture recognition. Specifically, this dataset consists of the real and augmented images. The real images captured in the simulated space station are closed to the real images in the space station environment. Meanwhile, we utilize the Mask-RCNN model to segment the foreground image including an astronaut from real data and capture the background images of the simulated space station at different views and illuminations. Then we combine them to synthesize the augmented images. A novel HA-SSD model consists of a lightweight backbone named MobileNet and a hierarchical attention mechanism. The MobileNet is used as the feature extractor, which contains several depth-wise separable convolutions to make a trade-off between latency and accuracy. Meanwhile, the hierarchical channel-wise attention module is adopted to exploit the fine semantic information to enrich the features for improving the performance of gesture recognition. We conduct experiments to demonstrate that our dataset is suitable for the space station and our approach is able to effectively localize and recognize the gestures with strong generality.

show abstract

Section: B Object Detection Methods Based On Deep Learning and Lightmentioning

confidence: 99%

Hierarchical Attention-Based Astronaut Gesture Recognition: A Dataset and CNN Model

Zhang

Wang

2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Action recognition [8] [9] classifies different actions based on observed action sequences. However, intention prediction [10] [11] [12] predicts the intention of an action from the subtle motion patterns of the same action. Figure 1 illustrates the difference between action recognition and intention prediction.…”

Section: Introductionmentioning

confidence: 99%

Untitled

2022

IJAIA

View full text Add to dashboard Cite

Data augmentation has been broadly applied in training deep-learning models to increase the diversity of data. This study ingestigates the effectiveness of different data augmentation methods for deep-learningbased human intention prediction when only limited training data is available.A human participant pitches a ball to nine potential targets in our experiment. We expect to predict which target the participant pitches the ball to. Firstly, the effectiveness of 10 data augmentation groups is evaluated on a single-participant data set using RGB images. Secondly, the best data augmentation method (i.e., random cropping) on the single-participant data set is further evaluated on a multi-participant data set to assess its generalization ability. Finally, the effectiveness of random cropping on fusion data of RGB images and optical flow is evaluated on both single-and multi-participant data sets. Experiment results show that: 1) Data augmentation methods that crop or deform images can improve the prediction performance; 2) Random cropping can be generalized to the multi-participant data set (prediction accuracy is improved from 50% to 57.4%); and 3) Random cropping with fusion data of RGB images and optical flow can further improve the prediction accuracy from 57.4% to 63.9% on the multi-participant data set.

show abstract