Deep Convolutional and LSTM Neural Network Architectures on Leap Motion Hand Tracking Data Sequences

Kritsis, Kosmas; Καλιακάτσος-Παπακώστας, Μάξιμος; Katsouros, Vassilis; Pikrakis, Aggelos

doi:10.23919/eusipco.2019.8902973

Cited by 9 publications

(2 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, a Convolutional Neural Network (CNN) was implemented in PyTorch [ 79 , 80 ] for classifying the tasks. CNN and its variations have been shown to be efficient algorithms for hand gesture classification [ 81 , 82 , 83 , 84 ]. The proposed architecture of the CNN is illustrated in Figure 9 .…”

Section: Methodsmentioning

confidence: 99%

Classification of Activities of Daily Living Based on Grasp Dynamics Obtained from a Leap Motion Controller

Sharif

Ahmadreza

Chembrammel

et al. 2022

Sensors

View full text Add to dashboard Cite

Stroke is one of the leading causes of mortality and disability worldwide. Several evaluation methods have been used to assess the effects of stroke on the performance of activities of daily living (ADL). However, these methods are qualitative. A first step toward developing a quantitative evaluation method is to classify different ADL tasks based on the hand grasp. In this paper, a dataset is presented that includes data collected by a leap motion controller on the hand grasps of healthy adults performing eight common ADL tasks. Then, a set of features with time and frequency domains is combined with two well-known classifiers, i.e., the support vector machine and convolutional neural network, to classify the tasks, and a classification accuracy of over 99% is achieved.

show abstract

Section: Methodsmentioning

confidence: 99%

Classification of Activities of Daily Living Based on Grasp Dynamics Obtained from a Leap Motion Controller

Sharif

Ahmadreza

Chembrammel

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…Research in human motion related problems, such as motion segmentation [33] and classification [34], [35], has greatly benefited from modern MoCap technology. Furthermore, the advent of publicly available image datasets with annotated 2D human body joints [17], [36], [37] and their correspondence to 3D human shapes [38]- [40], has paved the way for extensive improvements in the accuracy of automatic human pose estimation methods.…”

Section: Related Workmentioning

confidence: 99%

DanceConv: Dance Motion Generation With Convolutional Networks

et al. 2022

View full text Add to dashboard Cite

Automatically synthesizing dance motion sequences is an increasingly popular research task in the broader field of human motion analysis. Recent approaches have mostly used recurrent neural networks (RNNs), which are known to suffer from prediction error accumulation, usually limiting models to synthesize short choreographies of less than 100 poses. In this paper we present a multimodal convolutional autoencoder that combines 2D skeletal and audio information by employing an attention-based feature fusion mechanism, capable of generating novel dance motion sequences of arbitrary length. We first validate the ability of our system to capture the temporal context of dancing in a unimodal setting, by considering only skeletal features as input. According to 1440 rating answers provided by 24 participants in our initial user-study, we show that the optimal performance was presented by the model that was trained with input sequences of 500 poses. Based on this outcome, we train the proposed multimodal architecture with two different approaches, namely teacher-forcing and self-supervised curriculum learning, to deal with the autoregressive error accumulation phenomenon. In our evaluation campaign, we generate 1800 sequences and compare our method against two state-of-the-art approaches. Through qualitative and quantitative experiments we demonstrate the improvements introduced by the proposed multimodal architecture in terms of realism, motion diversity and multimodality, reducing the Fréchet Inception Distance (FID) metric value by 0.39. Subjective results confirm the effectiveness of our approach to synthesize diverse dance motion sequences, reporting a 6% increase in style consistency preference according to 1800 answers provided by 45 evaluators.

show abstract