In-air hand gesture signature recognition system based on 3-dimensional imagery

Khoh, Wee How; Pang, Ying Han; Teoh, Andrew Beng Jin

doi:10.1007/s11042-018-6458-7

Cited by 17 publications

(13 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They proposed a Gaussian distribution-based fusion algorithm to combine DTW, Fast Fourier Transform (FFT) and signature length analysis for the verification. Khoh et al [27] proposed an in-air signature acquisition method based on hand palm segmentation and gesture recognition using a single depth camera. Recently, Malik et al [11] proposed a deep learning-based in-air signature acquisition method using a single depth camera.…”

Section: Related Workmentioning

confidence: 99%

DeepAirSig: End-to-End Deep Learning Based in-Air Signature Verification

et al. 2020

View full text Add to dashboard Cite

In-air signature verification is vital for biometric user identification in contact-less mode. The state-of-the-art methods use heuristics for signature acquisition, and provide insufficient data to train neural networks for the verification. In this paper, we present a novel method for end-to-end deep learning based in-air signature verification using a depth sensor. In this regard, we propose a new medium-scale in-air signature dataset which is created using an accurate convolutional neural network (CNN) based 3D hand pose estimation algorithm. The proposed dataset offers a total of 1800 signatures collected from 40 subjects. So far, dynamic time warping (DTW) has been the most effective and commonly used method for verification. Keeping in view the significant advancement in deep learning, we present a more accurate deep learning based alternative which outperforms DTW by 67.6%. To this end, we train a personalized autoencoder to reconstruct the signature of each subject. Thereafter, the signature is verified by thresholding the reconstruction loss. We perform extensive experiments by formulating spatial and depth features of the signature in images and point clouds based representations. Moreover, for comparisons, we implement several deep learning algorithms (i.e. linear autoencoder, convolutional autoencoder, and Deep One-Class classifier). Our verification approach achieves an EER of 0.055%. The dataset is available at https://bit.ly/2mEJzOw.

show abstract

Section: Related Workmentioning

confidence: 99%

DeepAirSig: End-to-End Deep Learning Based in-Air Signature Verification

et al. 2020

View full text Add to dashboard Cite

show abstract

“…These number recognition methodologies are sensitive to any slight change in the angle of drawing and that causes a serious mismatching of the samples. More recent works [17], [18] in this category consist of high computational complexity pre-processing steps and extracting a set of features based on distance and angles followed by classification. They are sensitive to rotational changes of the shapes and not suitable for real time operation.…”

Section: In-air Handwritten Number Recognitionmentioning

confidence: 99%

Graph Spectral Domain Feature Learning With Application to in-Air Hand-Drawn Number and Shape Recognition

Alwaely

Abhayaratne

2019

IEEE Access

View full text Add to dashboard Cite

This paper addresses the problem of recognition of dynamic shapes by representing the structure in a shape as a graph and learning the graph spectral domain features. Our proposed method includes pre-processing for converting the dynamic shapes into a fully connected graph, followed by analysis of the eigenvectors of the normalized Laplacian of the graph adjacency matrix for forming the feature vectors. The method proposes to use the eigenvector corresponding to the lowest eigenvalue for formulating the feature vectors as it captures the details of the structure of the graph. The use of the proposed graph spectral domain representation has been demonstrated in an in-air hand-drawn number and symbol recognition applications. It has achieved average accuracy rates of 99.56% and 99.44%, for numbers and symbols, respectively, outperforming the existing methods for all datasets used. It also has the added benefits of fast real-time operation and invariance to rotation and flipping, making the recognition system robust to different writing and drawing variations.

show abstract

“…For the verification phase, the authors developed a fusion algorithm based on an improved DTW and the fast Fourier transform (FFT). Recently, Khoh et al [ 19 ] proposed a predictive palm segmentation algorithm to create a motion history image (MHI) using a depth sensor. Afterwards, they produced a two-dimensional representation of a hand-gesture signature based on the MHI.…”

Section: Related Workmentioning

confidence: 99%

“…This problem has not been well-addressed because the existing approaches try to locate only the fingertip using heuristics. Some of the approaches rely on palm center point tracking [ 17 , 19 ] which does not accurately mimic the pointing finger movement while signing in the air. Furthermore, due to their complex in-air signature acquisition systems, they are not suitable for real-time applications.…”

Section: Introductionmentioning

confidence: 99%

3DAirSig: A Framework for Enabling In-Air Signatures Using a Multi-Modal Depth Sensor

Malik

Elhayek

Ahmed

et al. 2018

Sensors

View full text Add to dashboard Cite

In-air signature is a new modality which is essential for user authentication and access control in noncontact mode and has been actively studied in recent years. However, it has been treated as a conventional online signature, which is essentially a 2D spatial representation. Notably, this modality bears a lot more potential due to an important hidden depth feature. Existing methods for in-air signature verification neither capture this unique depth feature explicitly nor fully explore its potential in verification. Moreover, these methods are based on heuristic approaches for fingertip or hand palm center detection, which are not feasible in practice. Inspired by the great progress in deep-learning-based hand pose estimation, we propose a real-time in-air signature acquisition method which estimates hand joint positions in 3D using a single depth image. The predicted 3D position of fingertip is recorded for each frame. We present four different implementations of a verification module, which are based on the extracted depth and spatial features. An ablation study was performed to explore the impact of the depth feature in particular. For matching, we employed the most commonly used multidimensional dynamic time warping (MD-DTW) algorithm. We created a new database which contains 600 signatures recorded from 15 different subjects. Extensive evaluations were performed on our database. Our method, called 3DAirSig, achieved an equal error rate (EER) of 0.46%. Experiments showed that depth itself is an important feature, which is sufficient for in-air signature verification.

show abstract

In-air hand gesture signature recognition system based on 3-dimensional imagery

Cited by 17 publications

References 30 publications

DeepAirSig: End-to-End Deep Learning Based in-Air Signature Verification

DeepAirSig: End-to-End Deep Learning Based in-Air Signature Verification

Graph Spectral Domain Feature Learning With Application to in-Air Hand-Drawn Number and Shape Recognition

3DAirSig: A Framework for Enabling In-Air Signatures Using a Multi-Modal Depth Sensor

Contact Info

Product

Resources

About