CNN Algorithm with SIFT to Enhance the Arabic Sign Language Recognition

Abstract

Sign language is used as a primary means of communication by millions of people who suffer from hearing problems. The unhearing people used visual language to interact with each other, Represented in sign language. There are features that the hearing impaired use to understand each other, which are difficult for normal people to understand. Therefore, deaf people will struggle to interact with society. This research aims to introduce a system for recognizing hand gestures in Arabic Sign Language (ArSL) through training the Convolutional Neural Network (CNN) on the images of ArSL gestures launched by the University of Prince Mohammad Bin Fahd, Saudi Arabia. A Scale Invariant Feature Transform (SIFT) algorithm is used for creating the feature vectors that contain shape, finger position, size, center points of palm, and hand margin by extracting the Important features for images of ArSL and transforming them to points of the vector. The accuracy of the proposed system is 97% using the SIFT with CNN, and equal to 94.8% nearly without SIFT. Finally, the proposed system was tried and tested on a group of persons and its effectiveness was proven after considering their observations.