Real-time sign language fingerspelling recognition using convolutional neural networks from depth map

Kang, Byeongkeun; Tripathi, Subarna; Nguyen, Truong Q.

doi:10.1109/acpr.2015.7486481

Cited by 97 publications

(37 citation statements)

References 13 publications

(18 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…As a basic task in hand research, hand segmentation contributes to many hand applications, such as pose estimation [8,9], gesture recognition [10,11] and so on.…”

Section: Related Workmentioning

confidence: 99%

DenseAttentionSeg: Segment hands from interacted objects using depth input

Zhang

Yong

et al. 2020

Applied Soft Computing

View full text Add to dashboard Cite

We propose a real-time DNN-based technique to segment hand and object of interacting motions from depth inputs. Our model is called DenseAttentionSeg, which contains a dense attention mechanism to fuse information in different scales and improves the results quality with skip-connections. Besides, we introduce a contour loss in model training, which helps to generate accurate hand and object boundaries. Finally, we propose and release our InterSegHands dataset 1 , a fine-scale hand segmentation dataset containing about 52k depth maps of hand-object interactions. Our experiments evaluate the effectiveness of our techniques and datasets, and indicate that our method outperforms the current state-of-the-art deep segmentation methods on interaction segmentation.

show abstract

“…As a basic task in hand research, hand segmentation contributes to many hand applications, such as pose estimation [8,9], gesture recognition [10,11] and so on.…”

Section: Related Workmentioning

confidence: 99%

DenseAttentionSeg: Segment hands from interacted objects using depth input

Zhang

Yong

et al. 2020

Applied Soft Computing

View full text Add to dashboard Cite

show abstract

“…Most researches of deep neural networks using depth maps treated a raw depth map as an image equivalent. For instance, a raw depth map was given as a direct input to the networks in hand pose estimation [25]- [27], human pose estimation [28], and fingerspelling recognition [29].…”

Section: Related Workmentioning

confidence: 99%

Depth-Adaptive Deep Neural Network for Semantic Segmentation

Kang

Lee

Nguyen

2018

IEEE Trans. Multimedia

Self Cite

View full text Add to dashboard Cite

Abstract-In this work, we present the depth-adaptive deep neural network using a depth map for semantic segmentation. Typical deep neural networks receive inputs at the predetermined locations regardless of the distance from the camera. This fixed receptive field presents a challenge to generalize the features of objects at various distances in neural networks. Specifically, the predetermined receptive fields are too small at a short distance, and vice versa. To overcome this challenge, we develop a neural network which is able to adapt the receptive field not only for each layer but also for each neuron at the spatial location. To adjust the receptive field, we propose the depthadaptive multiscale (DaM) convolution layer consisting of the adaptive perception neuron and the in-layer multiscale neuron. The adaptive perception neuron is to adjust the receptive field at each spatial location using the corresponding depth information. The in-layer multiscale neuron is to apply the different size of the receptive field at each feature space to learn features at multiple scales. The proposed DaM convolution is applied to two fully convolutional neural networks. We demonstrate the effectiveness of the proposed neural networks on the publicly available RGB-D dataset for semantic segmentation and the novel hand segmentation dataset for hand-object interaction. The experimental results show that the proposed method outperforms the state-of-the-art methods without any additional layers or pre/post-processing.

show abstract

“…Hand segmentation has been studied for many applications such as hand pose estimation [1][2][3][4][5][6], hand tracking [7][8][9], and gesture/sign/grasp recognition [10,11]. In color imagebased methods, skin color-based method has been popular [10,[12][13][14][15][16].…”

Section: Related Workmentioning

confidence: 99%

“…An alternative method is wearing a specific color glove [18]. For depth map-based methods, popular methods are using a wrist band [3,9,11] or using random decision forest (RDF) [1,2,19]. Although the method using a black wristband is uncomplicated and effective, it is inconvenient.…”

Section: Related Workmentioning

confidence: 99%

Hand segmentation for hand-object interaction from depth map

Kang

Tan²,

Jiang

et al. 2017

2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP)

Self Cite

View full text Add to dashboard Cite

Hand segmentation for hand-object interaction is a necessary preprocessing step in many applications such as augmented reality, medical application, and human-robot interaction. However, typical methods are based on color information which is not robust to objects with skin color, skin pigment difference, and light condition variations. Thus, we propose hand segmentation method for hand-object interaction using only a depth map. It is challenging because of the small depth difference between a hand and objects during an interaction. To overcome this challenge, we propose the two-stage random decision forest (RDF) method consisting of detecting hands and segmenting hands. To validate the proposed method, we demonstrate results on the publicly available dataset of hand segmentation for hand-object interaction. The proposed method achieves high accuracy in short processing time comparing to the other state-of-the-art methods.

show abstract

Real-time sign language fingerspelling recognition using convolutional neural networks from depth map

Cited by 97 publications

References 13 publications

DenseAttentionSeg: Segment hands from interacted objects using depth input

DenseAttentionSeg: Segment hands from interacted objects using depth input

Depth-Adaptive Deep Neural Network for Semantic Segmentation

Hand segmentation for hand-object interaction from depth map

Contact Info

Product

Resources

About