AI for Accessibility: Virtual Assistant for Hearing Impaired

Ozarkar, Saket; Chetwani, Raj; Devare, Sugam; Haryani, Sumeet; Giri, Nupur

doi:10.1109/icccnt49239.2020.9225392

Cited by 10 publications

(10 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Liang et al [117] Windows desktop British Dementia screening Zhou et al [118] iOS Hong Kong Translation Ozarkar et al [119] Android Indian Translation Joy et al [120] Android Indian Learning Paudyal et al [121] Android American Learning Luccio et al [122] Android Multiple Learning Chaikaew et al [123] Android, iOS Thai Learning Ku et al [124] -American Translation Potamianos et al [125] -Greek Learning Lee et al [126] -Korean Translation Schioppo et al [127] -American Learning Bansal et al [128] -American Learning Quandt et al [129] -American Learning…”

Section: Methods Operating System Sign Language Scenariomentioning

confidence: 99%

See 1 more Smart Citation

Artificial Intelligence Technologies for Sign Language

Papastratis

Chatzikonstantinou

Konstantinidis

et al. 2021

Sensors

View full text Add to dashboard Cite

AI technologies can play an important role in breaking down the communication barriers of deaf or hearing-impaired people with other communities, contributing significantly to their social inclusion. Recent advances in both sensing technologies and AI algorithms have paved the way for the development of various applications aiming at fulfilling the needs of deaf and hearing-impaired communities. To this end, this survey aims to provide a comprehensive review of state-of-the-art methods in sign language capturing, recognition, translation and representation, pinpointing their advantages and limitations. In addition, the survey presents a number of applications, while it discusses the main challenges in the field of sign language technologies. Future research direction are also proposed in order to assist prospective researchers towards further advancing the field.

show abstract

Section: Methods Operating System Sign Language Scenariomentioning

confidence: 99%

“…Moreover, the application does not run in real-time. On the other hand, Ozarkar et al in [ 119 ], implemented a smartphone application consisting of three modules. The sound classification module detected and classified input sounds and alerted the user through vibrations.…”

Section: Applicationsmentioning

confidence: 99%

Artificial Intelligence Technologies for Sign Language

Papastratis

Chatzikonstantinou

Konstantinidis

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…After developing and training the CNN model, the best performing Keras model was converted to TensorFlow.js to be deployed. Other examples of work that applies transfer learning include [89] which did transfer learning on InceptionV3, Resnet50, MobileNet, and [32] on PoseNet.…”

Section: Customizing Existing Models Using Transfer Learningmentioning

confidence: 99%

“…An object-based sound feedback system was developed in [85] that uses the SSD-MobileNetV2 object detection algorithm to assist visually impaired individuals in comprehending their environment via sound production. In [89], for Indian sign language recognition, transfer learning was used where pre-trained MobileNet model output features were fed to a KNN classifier to identify actions and predict respective words.…”

Section: Classification and Detection Appsmentioning

confidence: 99%

Front-end deep learning web apps development and deployment: a review

Goh

Abas

2022

Appl Intell

View full text Add to dashboard Cite

Machine learning and deep learning models are commonly developed using programming languages such as Python, C++, or R and deployed as web apps delivered from a back-end server or as mobile apps installed from an app store. However, recently front-end technologies and JavaScript libraries, such as TensorFlow.js, have been introduced to make machine learning more accessible to researchers and end-users. Using JavaScript, TensorFlow.js can define, train, and run new or existing, pre-trained machine learning models entirely in the browser from the client-side, which improves the user experience through interaction while preserving privacy. Deep learning models deployed on front-end browsers must be small, have fast inference, and ideally be interactive in real-time. Therefore, the emphasis on development and deployment is different. This paper aims to review the development and deployment of these deep-learning web apps to raise awareness of the recent advancements and encourage more researchers to take advantage of this technology for their own work. First, the rationale behind the deployment stack (front-end, JavaScript, and TensorFlow.js) is discussed. Then, the development approach for obtaining deep learning models that are optimized and suitable for front-end deployment is then described. The article also provides current web applications divided into seven categories to show deep learning potential on the front end. These include web apps for deep learning playground, pose detection and gesture tracking, music and art creation, expression detection and facial recognition, video segmentation, image and signal analysis, healthcare diagnosis, recognition, and identification.

show abstract

“…Furthermore, the programme is not real-time. In contrast, Ozarkar et al (Ozarkar-Chetwani, et al, 2020) developed a three-module smartphone application. The sound classification module recognised and classified input sounds, and vibrations notified the user.…”

Section: Related Studiesmentioning

confidence: 99%

Artificial Intelligence Biosensing System on Hand Gesture Recognition for the Hearing Impaired

Padmanandam¹,

Rajesh

Upadhyaya³

et al. 2022

International Journal of Operations Research and Information Systems

View full text Add to dashboard Cite

AI technologies have the potential to help deaf individuals communicate. Due to the complexity of sign fragmentation and the inadequacy of capturing hand gestures, the authors present a sign language recognition (SLR) system and wearable surface electromyography (sEMG) biosensing device based on a Deep SLR that converts sign language into printed message or speech, allowing people to better understand sign language and hand motions. On the forearms, two armbands containing a biosensor and multi-channel sEMG sensors are mounted to capture quite well arm and finger actions. Deep SLR was tested on an Android and iOS smartphone, and its usefulness was determined by comprehensive testing. Sign Speaker has a considerable limitation in terms of recognising two-handed signs with smartphone and smartwatch. To solve these issues, this research proposes a new real-time end-to-end SLR method. The average word error rate of continuous sentence recognition is 9.6%, and detecting signals and recognising a sentence with six sign words takes less than 0.9 s, demonstrating Deep SLR's recognition.

show abstract

AI for Accessibility: Virtual Assistant for Hearing Impaired

Cited by 10 publications

References 12 publications

Artificial Intelligence Technologies for Sign Language

Artificial Intelligence Technologies for Sign Language

Front-end deep learning web apps development and deployment: a review

Artificial Intelligence Biosensing System on Hand Gesture Recognition for the Hearing Impaired

Contact Info

Product

Resources

About