Fusion-Based Body-Worn IoT Sensor Platform for Gesture Recognition of Autism Spectrum Disorder Children

Abstract

The last decade’s developments in sensor technologies and artificial intelligence applications have received extensive attention for daily life activity recognition. Autism spectrum disorder (ASD) in children is a neurological development disorder that causes significant impairments in social interaction, communication, and sensory action deficiency. Children with ASD have deficits in memory, emotion, cognition, and social skills. ASD affects children’s communication skills and speaking abilities. ASD children have restricted interests and repetitive behavior. They can communicate in sign language but have difficulties communicating with others as not everyone knows sign language. This paper proposes a body-worn multi-sensor-based Internet of Things (IoT) platform using machine learning to recognize the complex sign language of speech-impaired children. Optimal sensor location is essential in extracting the features, as variations in placement result in an interpretation of recognition accuracy. We acquire the time-series data of sensors, extract various time-domain and frequency-domain features, and evaluate different classifiers for recognizing ASD children’s gestures. We compare in terms of accuracy the decision tree (DT), random forest, artificial neural network (ANN), and k-nearest neighbour (KNN) classifiers to recognize ASD children’s gestures, and the results showed more than 96% recognition accuracy.

Digital accessibility in the era of artificial intelligence—Bibliometric analysis and systematic review

“…. Additionally, a platform that recognizes complex sign language was designed to offer an effective means of communication for speechimpaired children (Ullah et al, 2023). The automatic recognition of two-handed signs in Indian Sign Language also serves as a teaching assistant, enhancing cognitive abilities and fostering interest in learning for hearing and speech-impaired children (Sreemathy et al, 2022).…”

“…Various studies have employed machine learning techniques to address accessibility challenges. For instance, Ullah et al (2023) utilized KNN, decision trees, random forest, and neural networks to decipher sign languages, underscoring the versatility of machine learning in linguistic contexts. A range of machine learning techniques, including Naïve Bayes, K-Nearest Neighbor, Stochastic Gradient Descent, Logistic Regression, Neural Networks, and Random Forest, have been used to classify facial expressions and jaw movements (Campomanes-Alvarez and Rosario Campomanes-Alvarez, 2021).…”

“…There is a paucity of comprehensive AI systems tailored to address the unique challenges faced by people with other disabilities such as speech and hearing impairments, autism spectrum disorder (ASD), neurological disorders, and motor impairments. While there are some noteworthy AI solutions for these other disability types (Zingoni et al, 2021;Ullah et al, 2023), the sheer volume of research and innovation predominantly dedicated to visual impairment underscores the need for a more equitable distribution of research efforts. It is crucial to expand the scope of AI-driven digital accessibility to bridge this gap and provide people with various disabilities with the same level of support, independence, and accessibility that the visually impaired enjoy.…”

Detection of autism spectrum disorder (ASD) in children and adults using machine learning

“…Application of intelligent approaches present advanced ways to economically detect ASD effected children and adults 44 . Models have been proposed in the literature describing application of different methods and approaches for ASD detection like structural MRI 45 , neural networks 46 , machine learning [47][48][49] , deep learning 50 , transfer learning 51,52 and IoT 53 . All these techniques have been applied to detect ASD with reasonable accuracy in children and adults but faced limitations of data acquisition as hospitals hesitate or refuse to share patient records due to organizational policies and regional data protection legislations.…”