A Comparison of Techniques for Sign Language Alphabet Recognition Using Armband Wearables

Abstract

Recent research has shown that reliable recognition of sign language words and phrases using user-friendly and noninvasive armbands is feasible and desirable. This work provides an analysis and implementation of including fingerspelling recognition (FR) in such systems, which is a much harder problem due to lack of distinctive hand movements. A novel algorithm called DyFAV (Dynamic Feature Selection and Voting) is proposed for this purpose that exploits the fact that fingerspelling has a finite corpus (26 alphabets for the American Sign Language (ASL)). Detailed analysis of the algorithm used as well as comparisons with other traditional machine-learning algorithms is provided. The system uses an independent multiple-agent voting approach to identify letters with high accuracy. The independent voting of the agents ensures that the algorithm is highly parallelizable and thus recognition times can be kept low to suit real-time mobile applications. A thorough explanation and analysis is presented on results obtained on the ASL alphabet corpus for nine people with limited training. An average recognition accuracy of 95.36% is reported and compared with recognition results from other machine-learning techniques. This result is extended by including six additional validation users with data collected under similar settings as the previous dataset. Furthermore, a feature selection schema using a subset of the sensors is proposed and the results are evaluated. The mobile, noninvasive, and real-time nature of the technology is demonstrated by evaluating performance on various types of Android phones and remote server configurations. A brief discussion of the user interface is provided along with guidelines for best practices.

A User-adaptive Modeling for Eating Action Identification from Wristband Time Series

“…Finally, we proposed a novel automatic segmentation technique for continuous recognition that can automatically annotate eating action. This technique can also be potentially used for other poly-componential gesture-based applications such as American Sign Language Recognition [28][29][30]. This is an important finding for future work.…”

Undergraduate Students of Special Education’s Readiness towards the Use of Information and Technology (ICT) in Teaching and Learning the Sign Language

“…In the study of Halawani (2008), Fajardo et al, (2009) and Paudyal et al, (2019), it is shown that teaching and learning sign language requires good ICT facilities to attract the individuals to learn. The availability of ICTs in the teaching and learning of sign language can enhance students' skills and quality (Paudyal et al, 2019). A conducive environment can be formed if the sign language courses have adequate ICT facilities (Chai et al, 2017).…”

“…According to Halawani (2008), Fajardo et al (2009) and Paudyal et al (2019), they also said that teaching and learning sign language requires good ICT facilities to attract individuals to learn them. In addition, teaching and learning sign language using ICT can enhance students' skills and quality (Paudyal et al, 2019).…”

Energy per Operation Optimization for Energy-Harvesting Wearable IoT Devices

“…Wearable IoT devices have been studied extensively due to their form factor and cost benefits. Researchers have proposed sensor networks, gesture-based control, health monitoring, and activity monitoring as potential applications of IoT devices [16][17][18][19][20][21][22]. Gesture recognition using wearable devices has received significant attention due to its applications in human-computer interaction, gesture-based control, and virtual reality [23][24][25][26].…”