Unified learning approach for egocentric hand gesture recognition and fingertip detection

Alam, Mohammad Mahmudul; Islam, Mohammad Tariqul; Rahman, S. M. Mahbubur

doi:10.1016/j.patcog.2021.108200

Cited by 35 publications

(18 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The algorithm first used the skin color threshold in the YCbCr color space to segment the hand region to obtain the hand contour, and then combined the local maximum value of the cumulative curvature and the detection of convex defects to detect the fingertip. • Fingertip detection based on YOLO [11].…”

Section: ) Test In Complex Backgroundsmentioning

confidence: 99%

“…et al [10] proposed to detect the hand area by the Faster R-CNN algorithm, and then use the CNN network to detect the fingertip position. In [11], the YOLO algorithm was used to obtain the hand area, and then the fingertip detection was regressed by the VGG16 full convolutional neural network. Y.-H.Chen et al [12] proposed an improved masked region convolutional neural network (Mask R-CNN), which uses a region-based CNN network to detect fingers and achieves fingertip detection through a three-layer CNN network.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Fingertip Detection Algorithm Based on Maximum Discrimination HOG Feature in Complex Background

Feng

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Section: ) Test In Complex Backgroundsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Fingertip Detection Algorithm Based on Maximum Discrimination HOG Feature in Complex Background

Feng

et al. 2023

IEEE Access

View full text Add to dashboard Cite

“…Further to the latest work-related hand gesture, the authors here [32] have applied deformable convolution neural networks. Fingerprint detection [33] for the recognition of hand gestures is another latest work proposed for HCI. A lightweight neural network [34] is applied for hand gesture recognition.…”

Section: Related Workmentioning

confidence: 99%

Sign Language Recognition for Arabic Alphabets Using Transfer Learning Technique

Zakariah

Alotaibi

Koundal

et al. 2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Sign language is essential for deaf and mute people to communicate with normal people and themselves. As ordinary people tend to ignore the importance of sign language, which is the mere source of communication for the deaf and the mute communities. These people are facing significant downfalls in their lives because of these disabilities or impairments leading to unemployment, severe depression, and several other symptoms. One of the services they are using for communication is the sign language interpreters. But hiring these interpreters is very costly, and therefore, a cheap solution is required for resolving this issue. Therefore, a system has been developed that will use the visual hand dataset based on an Arabic Sign Language and interpret this visual data in textual information. The dataset used consists of 54049 images of Arabic sign language alphabets consisting of 1500\ images per class, and each class represents a different meaning by its hand gesture or sign. Various preprocessing and data augmentation techniques have been applied to the images. The experiments have been performed using various pretrained models on the given dataset. Most of them performed pretty normally and in the final stage, the EfficientNetB4 model has been considered the best fit for the case. Considering the complexity of the dataset, models other than EfficientNetB4 do not perform well due to their lightweight architecture. EfficientNetB4 is a heavy-weight architecture that possesses more complexities comparatively. The best model is exposed with a training accuracy of 98 percent and a testing accuracy of 95 percent.

show abstract

“…There are two methods for identifying hand gestures for sign language translation: vision-based and glove-based [ 11 , 12 ]. Some of the latest works are on hand gesture recognition, which are recently published [ 9 , 13 , 14 ]. Electronic equipment such as data collection, accelerometers, bands, and other similar devices must be worn by participants in the contact-based method.…”

Section: Introductionmentioning

confidence: 99%

Human-Computer Interaction with Hand Gesture Recognition Using ResNet and MobileNet

Alnuaim

Zakariah

Hatamleh

et al. 2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Sign language is the native language of deaf people, which they use in their daily life, and it facilitates the communication process between deaf people. The problem faced by deaf people is targeted using sign language technique. Sign language refers to the use of the arms and hands to communicate, particularly among those who are deaf. This varies depending on the person and the location from which they come. As a result, there is no standardization about the sign language to be used; for example, American, British, Chinese, and Arab sign languages are all distinct. Here, in this study we trained a model, which will be able to classify the Arabic sign language, which consists of 32 Arabic alphabet sign classes. In images, sign language is detected through the pose of the hand. In this study, we proposed a framework, which consists of two CNN models, and each of them is individually trained on the training set. The final predictions of the two models were ensembled to achieve higher results. The dataset used in this study is released in 2019 and is called as ArSL2018. It is launched at the Prince Mohammad Bin Fahd University, Al Khobar, Saudi Arabia. The main contribution in this study is resizing the images to 64 ∗ 64 pixels, converting from grayscale images to three-channel images, and then applying the median filter to the images, which acts as lowpass filtering in order to smooth the images and reduce noise and to make the model more robust to avoid overfitting. Then, the preprocessed image is fed into two different models, which are ResNet50 and MobileNetV2. ResNet50 and MobileNetV2 architectures were implemented together. The results we achieved on the test set for the whole data are with an accuracy of about 97% after applying many preprocessing techniques and different hyperparameters for each model, and also different data augmentation techniques.

show abstract

Unified learning approach for egocentric hand gesture recognition and fingertip detection

Cited by 35 publications

References 9 publications

Fingertip Detection Algorithm Based on Maximum Discrimination HOG Feature in Complex Background

Fingertip Detection Algorithm Based on Maximum Discrimination HOG Feature in Complex Background

Sign Language Recognition for Arabic Alphabets Using Transfer Learning Technique

Human-Computer Interaction with Hand Gesture Recognition Using ResNet and MobileNet

Contact Info

Product

Resources

About