3DAirSig: A Framework for Enabling In-Air Signatures Using a Multi-Modal Depth Sensor

Abstract

In-air signature is a new modality which is essential for user authentication and access control in noncontact mode and has been actively studied in recent years. However, it has been treated as a conventional online signature, which is essentially a 2D spatial representation. Notably, this modality bears a lot more potential due to an important hidden depth feature. Existing methods for in-air signature verification neither capture this unique depth feature explicitly nor fully explore its potential in verification. Moreover, these methods are based on heuristic approaches for fingertip or hand palm center detection, which are not feasible in practice. Inspired by the great progress in deep-learning-based hand pose estimation, we propose a real-time in-air signature acquisition method which estimates hand joint positions in 3D using a single depth image. The predicted 3D position of fingertip is recorded for each frame. We present four different implementations of a verification module, which are based on the extracted depth and spatial features. An ablation study was performed to explore the impact of the depth feature in particular. For matching, we employed the most commonly used multidimensional dynamic time warping (MD-DTW) algorithm. We created a new database which contains 600 signatures recorded from 15 different subjects. Extensive evaluations were performed on our database. Our method, called 3DAirSig, achieved an equal error rate (EER) of 0.46%. Experiments showed that depth itself is an important feature, which is sufficient for in-air signature verification.

DeepAirSig: End-to-End Deep Learning Based in-Air Signature Verification

“…Khoh et al [27] proposed an in-air signature acquisition method based on hand palm segmentation and gesture recognition using a single depth camera. Recently, Malik et al [11] proposed a deep learning-based in-air signature acquisition method using a single depth camera. They employed a 3D hand pose estimation algorithm based on [28] for the acquisition but, their model is trained on a limited hand pose dataset.…”

“…Among these verification techniques, DTW has been the most effective and widely-used algorithm primarily due to its ability to well align the temporal signals [9]. Inair signatures have been acquired from egocentric viewpoint using Google Glass headset [10] or by placing a camera in front of the subject (i.e., 3rd person view) [11]- [14], or using FIGURE 1. We present an end-to-end deep learning-based in-air signature verification method.…”

“…For in-air signature verification, most of the algorithms (e.g. [10], [11], [14]) also employed DTW whereas, Fang et al [13] developed a novel fusion algorithm based on fast fourier transform (FFT), DTW and signature length analysis.…”

“…Most of the existing vision-based in-air signature verification methods [10], [13] record the signatures using an RGB sensor which does not capture an important hidden information in the 3rd dimension i.e., depth. Whereas a low cost and consumer-level depth sensor allows to effectively capture both the spatial and depth information [11]. All of the above mentioned state-of-the-art methods for in-air signature verification have used the conventional DTW algorithm for verification due to its accuracy.…”

“…For this reason, we propose an extended medium-scale dataset which comprises of 1800 signatures collected from 40 different subjects. The signatures are recorded using an effective 3D hand pose estimation algorithm [11] which we train on the state-ofthe-art million-scale hand pose dataset [17]. We formulate spatial and depth features of the signature trajectory in two different representations (i.e., Image-based and PointCloudbased representations).…”