Speech to Mexican Sign Language for Learning with an Avatar

Melchor, Fernando Barrera; Palacios, Julio César Alcibar; Pichardo-Lagunas, Obdulia; Martinez-Seis, Bella

doi:10.1007/978-3-030-60887-3_16

Cited by 6 publications

(6 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Second, in the use of avatars in sign language we find it in the study by Barrera et al (2020), which deals with the union of various techniques in the teaching of language and the use of graphic elements such as avatars for the simultaneous translation in Mexican sign language (MSL), taking into consideration the graphic character of the language itself in the implementation of the computer system, providing a translation based on rules associated with the historical linguistic learning of Mexico, for its implementation at the primary education level. It also addresses the grammatical structures in the teaching of history, using the data system through a cloud server processing natural language data to generate the transformation of data into voice and in turn, in the representation of gestures through a digitally implemented avatar.…”

Section: Integration Of Avatars As Translatorsmentioning

confidence: 74%

“…In the current panorama within the Mexican context we find two significant studies: on the one hand the use of "Leap Motion" and gamification in the teaching of sign language by Alvarez et al (2020) and on the other hand, the use of avatars by Barrera et al (2020), which provides simultaneous translation at the primary education level, storing data in the cloud.…”

Section: Integration Of Avatars As Translatorsmentioning

confidence: 98%

See 1 more Smart Citation

Sign language teaching technological advances and differences in international contexts

Prado

2021

IJILT

View full text Add to dashboard Cite

PurposeThe main objective of this article is to contribute to the field of accessibility in the teaching of sign language in the international panorama, examining its applicability, evaluation methods as well as the assistive technologies used to improve teaching experiences and the creation of new materials, proposing a theoretical framework that relates the teaching of sign language at different academic levels, the training of teachers, as well as parents, and the use of technology to achieve educational inclusion. It follows that the adoption of hybrid technology approaches, following universal design principles, can help to integrate access to education and sign language literacy.Design/methodology/approachThe working method used to carry out this work consists of a systematic review of the scientific literature. This is a research project based on the recapitulation of information about sign language in the international panorama and the improvements used for its instruction. For this purpose, the well-known PRISMA (Moher et al., 2009) is used to synthesize the search carried out. A systematic review of the articles published in scientific journals about pedagogy in the teaching of sign language in the international field and technological innovation for sign language teaching has been carried out, incorporating different approaches and personal assessments. The first phase of the method consists of identifying and analyzing the articles published in scientific journals on the teaching of sign language on the international scene and the importance of new educational models with the incorporation of various didactic adaptations, evaluating the selected articles over a period of time from 2009 to 2021. Six steps were used in the systematic review study (Figure 1). First, Steps 1–4 were conducted in 2020 and 2021 as part of a doctoral research. A schematic summary of Steps 1–4 is presented below, followed by Steps 5 and 6, added later to the study after completion of Step 4. In the case of the first four steps they were conducted by two researchers: the thesis tutor and the doctoral student, and Steps 5 and 6 were conducted by the Ph.FindingsTo carry out the analysis of the results, the codification of the variables was carried out. The selected studies are characterized by their international context with a final selection of 39 studies have found several variables that affect the relationship between the teaching of sign language in the international arena and the use of technological innovations to adapt their teaching to students (Figure 4). In this section we present the technology grouped variables included in each factor and the possibilities of standardization and applicability of sign language teaching in the international panorama. Within these articles, the importance of defining training programs in sign language for teachers and the need to evaluate teaching programs is identified, with a focus on actions to improve school curricula to achieve linguistic standardization and inclusion in the academic environment, as well as their use at all levels of education. Therefore, professional practices and cooperation between institutions such as: associations of families of deaf students and educational institutions must be improved, making it possible to give a quality education. Within the exclusion criteria, the articles that do not use sign language teaching tools (n = 45) were subsequently discarded, followed by the articles that do not indicate any intervention in students with disabilities (n = 48), concluding with the articles that nonsign language disability education (n = 44). Subsequently, among those selected, the technology articles that do not speak about sign language were discarded (n = 32), as well as the studies that protect indigenous language but not sign language (n = 33) and, to conclude, the linguistic normalization articles but do not cite sign language (n = 37).Originality/valueCommunication is the element by which the authors can understand each other with the rest of the people around us, in the diversity of language, within non-verbal language, the authors find sign language, the language of deaf and mute people, of families, as a professional employee and nowadays, learned by many non-deaf and mute people to achieve a more inclusive and integrated society with all people. Every country has the right to have its own sign language, especially one that claims its culture and customs, through non-verbal communication with which to express multiple meanings, emotions and intentions. It is essential to know and apply the technological advances that are being developed, promoting the right to autonomy and the defense of the indigenous language as a cultural element of the intangible heritage of each country. The use of technology allows the democratization of culture and access to information regardless of where one lives in the world, in an increasingly globalized society in which communication plays a fundamental role. In the case of the global pandemic, it has forced us to the advancement of home education and the use of efficient digital tools to achieve it such as videoconferencing, in the field of disability there are still many limitations on this use by the various companies that develop them. Within the inclusive educational research the authors must emphasize the need for equality of tools and content for all types of students, especially in sign language. Large digital gaps have been generated in families with and without resources at international level, which also have a member with a disability, this is pointed out in the research mentioning the current situation of the American continent, as well as the search for improvement of the tools and platforms in which they are developed.

show abstract

Section: Integration Of Avatars As Translatorsmentioning

confidence: 74%

Section: Integration Of Avatars As Translatorsmentioning

confidence: 98%

Sign language teaching technological advances and differences in international contexts

Prado

2021

IJILT

View full text Add to dashboard Cite

show abstract

“…Also presented is the SIGML sign language, where characters are presented using a 3D avatar, 3000 characters. The Mexican Sign Language avatar presented by the authors [33] was created based on a combination of natural language processing (NLP) techniques with the use of programming engines (Unity) to create animation. Using the structured language model of AZee, Paul's Avatar [34] was created, which is a hybrid system animated mainly with hand-made keyframes, and Kazoo's virtual avatar [35], which generates content from French to sign language.…”

Section: Translation Of Spoken Languages Into Sign Languagesmentioning

confidence: 99%

“…Access to translation tools enabling understanding of the content taught at school is invaluable for the intellectual and educational development of this group of people. The Mexican Sign Language Avatar, as a mobile application via a cloud server using NLP and automatic translation, will present limited content from a 4th grade primary Mexican history textbook in sign language [33].…”

Section: A Education Applicationmentioning

confidence: 99%

Sign language interpreting - relationships between research in different areas - overview

Probierz,

Kozak,

Piasecki

et al. 2023

Annals of Computer Science and Information Systems

View full text Add to dashboard Cite

Translation from the national language into sign language is an extremely important area of research and practice, which aims to ensure communication between deaf or hard of hearing people and the hearing community. The article provides an overview of the most important research on sign language interpretation conducted in various research areas. The latest scientific and theoretical achievements were presented, which contribute to a better understanding of the subject of sign language translation and the improvement of the quality of translation services. Our main goal is to identify outstanding areas of interdisciplinary research related to sign language translation and to identify links between these studies conducted in different areas. The conclusions of the article aim to broaden the knowledge and awareness of sign language translation and to identify areas that require further research and development. The work is linked to a project related to the application of machine learning in increasing accessibility for deaf people.

show abstract

“…The results indicate that avatar-based tutoring was more effective in assessing the child's knowledge of certain words in a sign language. 3D avatars are also being used to teach specific electrical engineering concepts in Portuguese Sign Language (LIBRAS) [30], to present content from a Mexican history textbook for elementary 4th grade in Mexican Sign Language [31] or to create digital math educational materials in American Sign Language and Arabic Sign Language [32,33]. Thanks to automated tools, it is also possible to learn the basics of sign language (ASL) on one's own.…”

Section: Introductionmentioning

confidence: 99%

Processing Real-Life Recordings of Facial Expressions of Polish Sign Language Using Action Units

Irasiak

Kozák

Piasecki

et al. 2023

Entropy

View full text Add to dashboard Cite

Automatic translation between the national language and sign language is a complex process similar to translation between two different foreign languages. A very important aspect is the precision of not only manual gestures but also facial expressions, which are extremely important in the overall context of a sentence. In this article, we present the problem of including facial expressions in the automation of Polish-to-Polish Sign Language (PJM) translation—this is part of an ongoing project related to a comprehensive solution allowing for the animation of manual gestures, body movements and facial expressions. Our approach explores the possibility of using action unit (AU) recognition in the automatic annotation of recordings, which in the subsequent steps will be used to train machine learning models. This paper aims to evaluate entropy in real-life translation recordings and analyze the data associated with the detected action units. Our approach has been subjected to evaluation by experts related to Polish Sign Language, and the results obtained allow for the development of further work related to automatic translation into Polish Sign Language.

show abstract

Speech to Mexican Sign Language for Learning with an Avatar

Cited by 6 publications

References 7 publications

Sign language teaching technological advances and differences in international contexts

Sign language teaching technological advances and differences in international contexts

Sign language interpreting - relationships between research in different areas - overview

Processing Real-Life Recordings of Facial Expressions of Polish Sign Language Using Action Units

Contact Info

Product

Resources

About