A person independent system for recognition of hand postures used in sign language

Kelly, Damien P.; McDonald, John; Markham, Charles H.

doi:10.1016/j.patrec.2010.02.004

Cited by 91 publications

(47 citation statements)

References 26 publications

(34 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…We can see that the best result for subjects 21-25 were 76% with 0.70 SIFT threshold, 80% with 0.70 SIFT threshold, 66% with 0.70 SIFT threshold, 68% with 0.65 and 0.70 SIFT threshold, and 62% with 0.75 SIFT threshold for the five subjects, respectively. Since the signers of this test set (subjects [21][22][23][24][25] were different signers from the training data set and the signature library, the results of this experiment provided low classification. Furthermore, when we used SIFT with the unconstrained system and complex natural backgrounds, the matched keypoints might be incorrectly matched, as shown Figure 3g.…”

Section: Resultsmentioning

confidence: 99%

A Novel String Grammar Unsupervised Possibilistic C-Medians Algorithm for Sign Language Translation Systems

2017

View full text Add to dashboard Cite

Abstract:Sign language is a basic method for solving communication problems between deaf and hearing people. In order to communicate, deaf and hearing people normally use hand gestures, which include a combination of hand positioning, hand shapes, and hand movements. Thai Sign Language is the communication method for Thai hearing-impaired people. Our objective is to improve the dynamic Thai Sign Language translation method with a video captioning technique that does not require prior hand region detection and segmentation through using the Scale Invariant Feature Transform (SIFT) method and the String Grammar Unsupervised Possibilistic C-Medians (sgUPCMed) algorithm. This work is the first to propose the sgUPCMed algorithm to cope with the unsupervised generation of multiple prototypes in the possibilistic sense for string data. In our experiments, the Thai Sign Language data set (10 isolated sign language words) was collected from 25 subjects. The best average result within the constrained environment of the blind test data sets of signer-dependent cases was 89-91%, and the successful rate of signer semi-independent cases was 81-85%, on average. For the blind test data sets of signer-independent cases, the best average classification rate was 77-80%. The average result of the system without a constrained environment was around 62-80% for the signer-independent experiments. To show that the proposed algorithm can be implemented in other sign languages, the American sign language (RWTH-BOSTON-50) data set, which consists of 31 isolated American Sign Language words, is also used in the experiment. The system provides 88.56% and 91.35% results on the validation set alone, and for both the training and validation sets, respectively.

show abstract

Section: Resultsmentioning

confidence: 99%

A Novel String Grammar Unsupervised Possibilistic C-Medians Algorithm for Sign Language Translation Systems

2017

View full text Add to dashboard Cite

show abstract

“…there is no need of gloves or markers like in [13,14,34] or accelerometers like in [5]) real-time approach to the detection of intuitive motion based gestures usable in different application contexts. The learning phase of our approach does not need the capture of ground-truth real data, since the patterns are defined synthetically by using a human arm model (see Section 5.1) making it is user independent (differently to [36,37,5,15]).…”

Section: Related Workmentioning

confidence: 99%

“…Other contexts of application of this approach can be the control of multimedia menus [31] or the point of view on a virtual environment. Other motion based gestures recognition could allow the interpretation of sign languages [9,13].…”

Section: Introductionmentioning

confidence: 99%

Real-time Motion-based Hand Gestures Recognition from Time-of-Flight Video

Molina

Pajuelo

Martínez

2015

J Sign Process Syst

View full text Add to dashboard Cite

Esta es la versión de autor de la comunicación de congreso publicada en: This is an author produced version of a paper published in: Abstract This paper presents an innovative solution based on Time-Of-Flight (TOF) video technology to motion patterns detection for real-time dynamic hand gesture recognition. The resulting system is able to detect motion-based hand gestures getting as input depth images. The recognizable motion patterns are modeled on the basis of the human arm anatomy and its degrees of freedom, generating a collection of synthetic motion patterns that is compared with the captured input patterns in order to finally classify the input gesture. For the evaluation of our system a significant collection of gestures has been compiled, getting results for 3D pattern classification as well as a comparison with the results using only 2D information.

show abstract

“…La comunicación no verbal en personas con discapacidad auditiva involucra una serie de aspectos relacionados con los movimientos gestuales de la mano como son las posturas, en donde se encuentran la forma y la orientación de la mano, o los gestos temporales, los cuales se relacionan con su movimiento y posición [9,10], y de manera general con los estados de transición y las posturas del cuerpo humano [11]. Para el reconocimiento de un movimiento gestual desde las posturas y los gestos temporales, se han desarrollado una serie de trabajos basados en el procesamiento digital de imágenes sobre video, los cuales reconocen un movimiento gestual en tiempo real, mediante la utilización de técnicas como la transformada wavelet, o las redes neuronales por aprendizaje no supervisado [11].…”

unclassified

“…Para el reconocimiento de un movimiento gestual desde las posturas y los gestos temporales, se han desarrollado una serie de trabajos basados en el procesamiento digital de imágenes sobre video, los cuales reconocen un movimiento gestual en tiempo real, mediante la utilización de técnicas como la transformada wavelet, o las redes neuronales por aprendizaje no supervisado [11]. Para el caso del reconocimiento de movimientos gestuales a partir de imágenes que describen trayectorias sobre puntos específicos ubicados en la mano, se han utilizado sistemas como el Kinect Motion de Microsoft (www.xbox.com/es-ES/kinect), o modelos que utilizan regresiones no lineales múltiples para la descripción de dichas trayectorias en el espacio [12].…”

unclassified

Traducción automática del lenguaje dactilológico de sordos y sordomudos mediante sistemas adaptativos (AutomAtic trAnslAtion of the dActilologic language of hearing impaired by adaptive systems)

Palacio

Betancur²,

Gómez³

2014

RIB

View full text Add to dashboard Cite

AutomAtic trAnslAtion of the dActilologic lAnguAge of heAring impAired by AdAptive systemsResumen ─ Una de las principales limitaciones que presentan las personas con discapacidad auditiva está directamente relacionada con su dificultad para interactuar con otras personas, ya sea de forma verbal o a través de sistemas auxiliares basados en la voz y el audio. En este artículo se presenta el desarrollo de un sistema integrado de hardware y software, para el reconocimiento automático del lenguaje dactilológico de señas utilizado por personas con este tipo de discapacidad. El hardware está compuesto por un sistema inalámbrico adherido a un guante, el cual posee un conjunto de sensores que capturan una serie de señales generadas por los movimientos gestuales de la mano, y un modelo por adaptación basado en los principios de la computación neuronal, el cual permite su reconocimiento en términos de un lenguaje dactilológico en particular. Los resultados arrojados por el sistema integrado mostraron gran efectividad en el reconocimiento de las vocales que conforman el lenguaje dactilológico en español, esto gracias a la capacidad que posee el modelo de asociar un conjunto de señales de entrada, con un movimiento dactilológico en particular.Palabras clave ─ Computación neuronal híbrida, lenguaje dactilológico, Protocolo ZigBee, discapacidad auditiva, sistemas de reconocimiento.Abstract ─ One of the main limitations of the people with hearing impairment is directly related to their difficulty interacting with others, either verbally or through auxiliary systems based on voice and audio. This paper presents the development of an integrated system of hardware and software for automatic fingerspelling sign language used by people with this type of disability. The hardware system comprises a glove which has a set of wireless sensors that capture a series of signals generated by the hand gestures, and a adaptive model based on the principles of neural computation, that allows recognition of a particular dactilologic language. Results from the integrated system showed great effectiveness in recognizing vowels from the dactilologic Spanish language. This recognition was influenced by the dimensionality reduction made by the neural model of the input signals representing movements, and the sensitivity factor that sets the limit between recognition and learning.

show abstract

A person independent system for recognition of hand postures used in sign language

Cited by 91 publications

References 26 publications

A Novel String Grammar Unsupervised Possibilistic C-Medians Algorithm for Sign Language Translation Systems

A Novel String Grammar Unsupervised Possibilistic C-Medians Algorithm for Sign Language Translation Systems

Real-time Motion-based Hand Gestures Recognition from Time-of-Flight Video

Traducción automática del lenguaje dactilológico de sordos y sordomudos mediante sistemas adaptativos (AutomAtic trAnslAtion of the dActilologic language of hearing impaired by adaptive systems)

Contact Info

Product

Resources

About