Privacy-Preserving British Sign Language Recognition Using Deep Learning

Abstract

Sign language is a mean of communication between the deaf community and hearing people, who use hand gestures, facial expressions, and body language to communicate. It has the same level of complexity as spoken language, but it does not employ the same sentence structure as English. The motions in sign language are made up of a range of distinct hand and finger articulations that are occasionally synchronized with the head, face, and body. Existing sign language recognition systems are mainly camera-based, which have fundamental limitations of poor lighting conditions, potential training challenges with longer video sequence data, and serious privacy concerns. This study presents a contact-less and privacy-preserving British sign language (BSL) Recognition system using Radar and deep learning algorithms, namely Inceptionv3, VGG16, and VGG19. The six most common emotions are considered, namely confused, depressed, happy, hate, lonely, and sad. The collected data is represented in the form of spectrograms. The deep learning models, InceptionV3, VGG19, and VGG16 then extract spatiotemporal features from the Spectrogram. Finally, the BSL emotions are accurately identified by classifying the Spectrograms, into the considered emotions signs. The simulation results demonstrate that a maximum classifying accuracy of 93.33% is obtained using VGG16.<br>

Radio frequency sensing and its innovative applications in diverse sectors: A comprehensive study

“…The spectrogram divided into test and train was fed into a pretrained deep learning algorithm. The results show that VGG16 outperformed VGG19 and InceptionV3 with around 93.33% accuracy (Hameed et al, 2022b).…”

RF-CSign: A Chinese Sign Language Recognition System Based on Large Kernel Convolution and Normalization-Based Attention

“…In addition, wireless technologies like Wi-Fi [15], [19], RFID [20], and radar [8], [11], [33], [34] have been used for sign language recognition. For instance, the channel state information of Wi-Fi was used to collect data for the recognition of American Sign Language within the contexts of SignFi [15] and WiSign [19].…”

“…For instance, the channel state information of Wi-Fi was used to collect data for the recognition of American Sign Language within the contexts of SignFi [15] and WiSign [19]. Related literature has also revealed the use of UWB Radar to collect data on British Sign Language and the use of VGG16 model to recognize six sign languages that express emotions [11]. There was also the use of CW Radar to collect spectrograms of sign language signals, in which five sign languages were classified based on the KNN algorithm [33].…”

“…This section compares the accuracy of the sign language recognition model of RF-CSign with five sign language recognition models: ResNet-18 [44], ResNet-34 [44], Tgt [47], TL [46], ResNet-6 [48], and VGG-16 [11]. These models were trained on the RFID sign language dataset and the accuracy of each validation set was recorded.…”

“…Approaches to sign language recognition can be classified as vision-based [9], [10], sensor-based [11], [12], [13], [14], [15], [16], [17], and a blend of both [18]. Vision-based gesture recognition techniques use a camera to capture the visual data of a gesture, and then process the visual data to complete the recognition.…”