Simple and effective deep hand shape and pose regression from a single depth image

Malik, Jameel; Elhayek, Ahmed; Nunnari, Fabrizio; Stricker, Didier

doi:10.1016/j.cag.2019.10.002

Cited by 19 publications

(11 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We propose a new medium-scale dataset which is adequate to train a neural network based approach for verification. Recent years have seen a significant progress in 3D hand pose estimation from a single depth image [29]- [32]. However, most of the existing methods either suffer from low accuracy [33]- [36] or they do not provide sufficient runtime performance [29], [30], [37], [38] which restrict their direct usage for a realtime application.…”

Section: Related Workmentioning

confidence: 99%

DeepAirSig: End-to-End Deep Learning Based in-Air Signature Verification

et al. 2020

Self Cite

View full text Add to dashboard Cite

In-air signature verification is vital for biometric user identification in contact-less mode. The state-of-the-art methods use heuristics for signature acquisition, and provide insufficient data to train neural networks for the verification. In this paper, we present a novel method for end-to-end deep learning based in-air signature verification using a depth sensor. In this regard, we propose a new medium-scale in-air signature dataset which is created using an accurate convolutional neural network (CNN) based 3D hand pose estimation algorithm. The proposed dataset offers a total of 1800 signatures collected from 40 subjects. So far, dynamic time warping (DTW) has been the most effective and commonly used method for verification. Keeping in view the significant advancement in deep learning, we present a more accurate deep learning based alternative which outperforms DTW by 67.6%. To this end, we train a personalized autoencoder to reconstruct the signature of each subject. Thereafter, the signature is verified by thresholding the reconstruction loss. We perform extensive experiments by formulating spatial and depth features of the signature in images and point clouds based representations. Moreover, for comparisons, we implement several deep learning algorithms (i.e. linear autoencoder, convolutional autoencoder, and Deep One-Class classifier). Our verification approach achieves an EER of 0.055%. The dataset is available at https://bit.ly/2mEJzOw.

show abstract

Section: Related Workmentioning

confidence: 99%

DeepAirSig: End-to-End Deep Learning Based in-Air Signature Verification

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…Different methods utilized for prediction are applied on only skeletons, our emphasis is to get as output a condensed hand mesh that can gather relations with objects. Very recently, Panteleris et al [6] and Malik et al [7] have produced full hand meshes. However, [8] achieves this as a post-processing step by fitting it to 2D predictions.…”

Section: Related Workmentioning

confidence: 99%

“…Vision-based hand pose estimation and tracking have been extensively studied over many years. The successes in performance in this field have been mainly due to two dominating trends: depth image and deep learning [7]. The problem of 3D hand pose tracking has been mainstreaming among researchers of computer vision, as it plays an important role in human-computer interaction such as virtual/augmented reality applications.…”

Section: Related Workmentioning

confidence: 99%

3D Hand Gestures Segmentation and Optimized Classification Using Deep Learning

et al. 2021

View full text Add to dashboard Cite

Hand gestures recognition system has massive applications which are mainly utilized in robotics and computer vision specially to control Unmanned Aerial Vehicles (UAV). These methods bypass the presence of electronic control to UAVs and provide an ease to the operators. In this paper, we present a method for 3D hand gestures segmentation and classification by combining MASK-RCNN with Grass Hopper Optimization. We created a private 3D and RGB hand gestures dataset using Intel Kinetic and Intel Real sense d435i camera, then proposed a model for RGB hand gestures to estimate the key points using human kinematics, the key points later then utilize to get the best degree of freedom (DoF). The grass hopper optimization besides minimum distance function was applied to achieve the finest deep features from the 3D hand gestures dataset. The ResNet50 network is used as the backbone to calculate the Overlap Coefficient (OC) for segmentation and the ResNet50, ResNet101 networks to calculate the classification for 3D hand gestures. The classification accuracy achieved on the private dataset is 99.05% and 99.29% on public Microsoft Kinect and Leap Motion dataset where the OC are 88.16%. and 88.19% respectively.

show abstract

“…To solve the 2-d hand pose estimation problem in egocentric perspective, people annotated real data [22,19,7] and created synthetic data [19,18,15,16]. All of them provide 3-d hand pose ground truth.…”

Section: Related Workmentioning

confidence: 99%

Understanding Egocentric Hand-Object Interactions from Hand Pose Estimation

Lu,

Mayol-Cuevas

2021

Preprint

View full text Add to dashboard Cite

In this paper, we address the problem of estimating the hand pose from the egocentric view when the hand is interacting with objects. Specifically, we propose a method to label a dataset Ego-Siam which contains the egocentric images pair-wisely. We also use the collected pairwise data to train our encoder-decoder style network which has been proven efficient in [31]. This could bring extra training efficiency and testing accuracy. Our network is light weight and can be performed with over 30 FPS with an outdated GPU. We demonstrate that our method outperforms Mueller et al. [18] which is the state of the art work dealing with egocentric hand-object interaction problem on the GANerated [19] dataset. To show the ability to preserve the semantic information of our method, we also report the performance of grasp type classification on dataset and outperforms the benchmark with only using predicted 3-d hand pose.

show abstract

Simple and effective deep hand shape and pose regression from a single depth image

Cited by 19 publications

References 6 publications

DeepAirSig: End-to-End Deep Learning Based in-Air Signature Verification

DeepAirSig: End-to-End Deep Learning Based in-Air Signature Verification

3D Hand Gestures Segmentation and Optimized Classification Using Deep Learning

Understanding Egocentric Hand-Object Interactions from Hand Pose Estimation

Contact Info

Product

Resources

About