Development of an American Sign Language Recognition System using Canny Edge and Histogram of Oriented Gradient

Abstract

Sign language is used by people who have hearing and speaking difficulties, but not understood by many without these difficulties. Therefore, sign language recognition systems are developed to aid communication between hearing impaired people and others. This paper developed a static American Sign Language Recognition (ASLR) system using canny-edge and histogram of oriented gradient (HOG) for feature extraction with K-Nearest Neighbour (K-NN) as classifier. The sign language image datasets used consist of English alphabets from both Massey University and Kaggle, and numbers (0-9) from Massey University. Median filter was used to remove noise after images were converted to grayscale. Otsu algorithm was used for segmentation while edges in the images were preserved using canny edge detection technique with HOG parameters tuning to obtain feature vectors. The extracted features were used by K-NN for classification. An average recognition accuracy and computational testing time of 97.6% and 0.39s respectively were obtained based on experiments with the Massey University dataset. Similarly, an average recognition accuracy and computational testing time of 99.0% and 0.43s respectively were obtained based on experiments with the Kaggle dataset. The developed system successfully recognized static English alphabets and numbers and outperformed some existing systems.

Enhancing Object Detection for VIPs Using YOLOv4_Resnet101 and Text-to-Speech Conversion Model

“…In their work, 50% of the images were used for training and validation, and 50% were used for testing. YOLOv4 for the facilitation of VIPs in outdoor environments was employed in [15]. They used a custom dataset of two classes: gutter and bollard.…”

“…The Edge Box-SSD technique may not perform well in increasingly complicated and high-resolution situations seen in real-world object identification applications. Adeyanju et al [15] applied CNN for object detection on a custom dataset consisting of two classes. They obtained a high accuracy of 84%.…”

Fusion of Attention-Based Convolution Neural Network and HOG Features for Static Sign Language Recognition

“…since there are two branches of the CNN model for feature extraction for each layer of the model. In[31], the authors have used fused features of pre-trained VGG16 and attention-based VGG16.The model has a computational complexity of 2 * O (∑ L i=1 S 2 L * K 2 L * C L−1 * C L + ∑ f i=1 F f ) +O(k * h).The study[32] involves canny edge detection, HOG feature extraction, and KNN for classification, which has an overall time complexity of O (m * n log m * n) + O (n 2 ) + O(d * F), where m * n is the image size, d, and F is the number of data points and features, respectively. In [34], authors have used a vision transformer encoder with a time complexity of O (8 * s 2 * d) + O (8 * s * d 2 ), where s is the sequence length and d is the depth.…”