Hybrid paradigm for Spanish Sign Language synthesis

Abstract

This work presents a hybrid approach to sign language synthesis. This approach allows hand-tuning the phonetic description of the signs, focusing on the time aspect of the sign. Therefore, we keep the capacity of performing morphonological operations, like the notation-based approaches and improving the synthetic signing performance, like the hand-tuned animations approach.Our approach simplifies input message description using a new high level notation and storing sign phonetic descriptions in a relational database. The relational database allows more flexible sign phonetic descriptions; it also allows describing sign timing and the synchronization between sign phonemes. The new notation, named HLSML, focuses on message description; it is a gloss-based notation. HLSML introduces several tags that allow modifying the signs in the message defining dialect and mood variations (both defined in the relational database) and message timing (transition durations and pauses). We also propose a new avatar design that simplifies the development of the synthesizer and avoids any interference in the independence of the sign language phonemes during the animation.The obtained results show an increase of the sign recognition rate compared to other approaches. This improvement is based on the active role that the sign language experts have in the description of signs, allowed by the flexibility of the sign storage approach. The approach will simplify the description of synthesizable signed messages so the creation of multimedia signed contents will be easier.

Evolution and Trends in Sign Language Avatar Systems: Unveiling a 40-Year Journey via Systematic Review

“…Ref. [59] adopted a skeleton-based animation approach with modifications to enable independent wrist orientation and position. Some papers adopted previously available avatars into their research.…”

“…Ref. [59] developed a relational database with four logical levels, where the first level served as a gloss dictionary, and the subsequent levels described the signs in phonetic parameters, phonemes, allophones, and hand orientation and movements. The database allowed for storing different dialect variations and mood realizations of the same concept.…”

“…Mocap technology was commonly used, as seen in [41,43,46,58,73,84]. Facial expression using face animation parameters was another technique used, meaning that the avatar's facial features were a set of polygons and they were controlled by manipulating the facial parameters [42,45,52,59]. Scripting engines such as Maxine have also been used to generate facial animations [53].…”

KAZOO: a sign language generation platform based on production rules

“…Some researchers advocate the need to adopt other points of view, specific to SLs, to overcome the limitations inherent to the classical approaches, especially on the multilinearity of gestural activity. Some models account for parallel gestural events, carried by different articulators over different time spans [10,16,20]. This was the reason for designing a platform, grounded on these SL-specific linguistic models.…”

Building machine-readable knowledge representations for Turkish sign language generation

“…The recent approaches tend to concatenate bigger chunks, which leads to more naturally sounding speech. As a result, some recent studies focus on gloss-based notations (HLMSL for Spanish [21,11,22], [23] for ASL) as we do in our proposed scheme.…”