A Web-Based Sign Language Translator Using 3D Video Processing

Li, Kin Fun; Lothrop, Kylee; Gill, Ethan; Lau, Stephen L. H.

doi:10.1109/nbis.2011.60

Cited by 41 publications

(10 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Li [12] also worked with sentence recognition of ASL and used the Kinect sensor to acquire data. The feature vector was based on the joints of the body calculated by the Kinect and the author used template matching technique for comparing the signals.…”

Section: Related Workmentioning

confidence: 99%

LIBRAS Sign Language Hand Configuration Recognition Based on 3D Meshes

Porfírio

Wiggers

Oliveira

et al. 2013

2013 IEEE International Conference on Systems, Man, and Cybernetics

View full text Add to dashboard Cite

This paper presents a method for recognizing hand configurations of the Brazilian sign language (LIBRAS) using 3D meshes and 2D projections of the hand. Five actors performing 61 different hand configurations of the LIBRAS language were recorded twice, and the videos were manually segmented to extract one frame with a frontal and one with a lateral view of the hand. For each frame pair, a 3D mesh of the hand was constructed using the Shape from Silhouette method, and the rotation, translation and scale invariant Spherical Harmonics method was used to extract features for classification. A Support Vector Machine (SVM) achieved a correct classification of Rank1 = 86.06% and Rank3 = 96.83% on a database composed of 610 meshes. SVM classification was also performed on a database composed of 610 image pairs using 2D horizontal and vertical projections as features, resulting in Rank1 = 88.69% and Rank3 = 98.36%. Results encourage the use of 3D meshes as opposed to videos or images, given that their direct, real time acquisition is becoming possible due to devices like LeapMotion R or high resolution depth cameras.

show abstract

Section: Related Workmentioning

confidence: 99%

LIBRAS Sign Language Hand Configuration Recognition Based on 3D Meshes

Porfírio

Wiggers

Oliveira

et al. 2013

2013 IEEE International Conference on Systems, Man, and Cybernetics

View full text Add to dashboard Cite

show abstract

“…Raheja et al [16] realized hand detection by segmenting Kinect's distance and depth vector. Li et al [17] recognized sign language with Kinect, and displayed the corresponding meaning of action in the interactive interface. Xia et al [18] used the Kinect depth image and the Canny operator to extract the data of the human body edge to detect and track human body.…”

Section: Kinect-based Interactionmentioning

confidence: 99%

Posture Recognition Technology Based on Kinect

Chu

Xin

2020

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

Aiming at the complexity of posture recognition with Kinect, a method of posture recognition using distance characteristics is proposed. Firstly, depth image data was collected by Kinect, and threedimensional coordinate information of 20 skeleton joints was obtained. Secondly, according to the contribution of joints to posture expression, 60 dimensional Kinect skeleton joint data was transformed into a vector of 24-dimensional distance characteristics which were normalized according to the human body structure. Thirdly, a static posture recognition method of the shortest distance and a dynamic posture recognition method of the minimum accumulative distance with dynamic time warping (DTW) were proposed. The experimental results showed that the recognition rates of static postures, non-cross-subject dynamic postures and cross-subject dynamic postures were 95.9%, 93.6% and 89.8% respectively. Finally, posture selection, Kinect placement, and comparisons with literatures were discussed, which provides a reference for Kinect based posture recognition technology and interaction design.

show abstract

“…We conclude that the 2D/RGB video studies we reviewed, recognizing gestures or fingerspelled letter signs, experienced increased algorithm complexity, decreases in both recognition time and accuracy. Hand and arm joints used for feature extraction Reference [16] focused on Kinect's 3D/IR photo sensor data of captured user joints. The matched signs were translated into words or phrases.…”

Section: Edge Detection and Contours For Feature Extractionmentioning

confidence: 99%

“…Reference [16] focused on the complete human body skeleton for gestures, which is described by the author as the signing space (the area from top of head to the waist and full arm extension on both sides of the torso) was out of the scope that our study focused on; i.e. the five ACHLS.…”

Section: D/ir Photomentioning

confidence: 99%

Precise Motif Sequence Pattern Recognition of American Sign Language (ASL) Letter Signs

Kutscher¹,

Qu²

2016

International Journal of Signal Processing Systems IJSPS

View full text Add to dashboard Cite

Unsuccessful pattern recognition of the complete set of fingerspelled letter signs were reported by studies using specialized video, RGB video, and Infrared (IR) video cameras, combined with various technologies. One reviewed study scaled (resized) letters to a specific size for ease of pattern recognition. Our study used the five similarly contoured ASL closed hand letter signs (ACLHS) A, M, N, S and T, with differentiation problems due to similar contours to show that motif sequences, formulated from the unique signatures of each of the five ACHLS, are less complex and faster at pattern recognition than scaling each captured letter sign dynamically. Thus, IR photo sensor data can be rotated to specific targets, creating unique signature patterns for each of the five ACHLS, and those unique signatures can be formulated into motif sequences for consistent and accurate pattern recognition of unknown ACHLS, regardless of similar handshape contours. 

show abstract

A Web-Based Sign Language Translator Using 3D Video Processing

Cited by 41 publications

References 9 publications

LIBRAS Sign Language Hand Configuration Recognition Based on 3D Meshes

LIBRAS Sign Language Hand Configuration Recognition Based on 3D Meshes

Posture Recognition Technology Based on Kinect

Precise Motif Sequence Pattern Recognition of American Sign Language (ASL) Letter Signs

Contact Info

Product

Resources

About