Evaluating the Efficiency of CBAM-Resnet Using Malaysian Sign Language

Khan, Rehman Ullah; Wong, Woei Sheng; Ullah, Insaf; Algarni, Fahad; Haq, Muhammad Inam Ul; Barawi, Mohamad Hardyman; Khan, Muhammad Asghar

doi:10.32604/cmc.2022.022471

Cited by 4 publications

(3 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then, object detection is applied to the segmented tea buds to identify the picking points, followed by the final localization of the tea buds. The addition of an attention mechanism module is an effective way to enhance model performance (Chen et al, 2022;Khan et al, 2022). The proposed improved DeepLabV3+ model based on attention mechanism can effectively accomplish the segmentation task.…”

Section: Discussionmentioning

confidence: 99%

A tea bud segmentation, detection and picking point localization based on the MDY7-3PTB model

Zhang,

Sun,

Xie

et al. 2023

Front. Plant Sci.

View full text Add to dashboard Cite

IntroductionThe identification and localization of tea picking points is a prerequisite for achieving automatic picking of famous tea. However, due to the similarity in color between tea buds and young leaves and old leaves, it is difficult for the human eye to accurately identify them.MethodsTo address the problem of segmentation, detection, and localization of tea picking points in the complex environment of mechanical picking of famous tea, this paper proposes a new model called the MDY7-3PTB model, which combines the high-precision segmentation capability of DeepLabv3+ and the rapid detection capability of YOLOv7. This model achieves the process of segmentation first, followed by detection and finally localization of tea buds, resulting in accurate identification of the tea bud picking point. This model replaced the DeepLabv3+ feature extraction network with the more lightweight MobileNetV2 network to improve the model computation speed. In addition, multiple attention mechanisms (CBAM) were fused into the feature extraction and ASPP modules to further optimize model performance. Moreover, to address the problem of class imbalance in the dataset, the Focal Loss function was used to correct data imbalance and improve segmentation, detection, and positioning accuracy.Results and discussionThe MDY7-3PTB model achieved a mean intersection over union (mIoU) of 86.61%, a mean pixel accuracy (mPA) of 93.01%, and a mean recall (mRecall) of 91.78% on the tea bud segmentation dataset, which performed better than usual segmentation models such as PSPNet, Unet, and DeeplabV3+. In terms of tea bud picking point recognition and positioning, the model achieved a mean average precision (mAP) of 93.52%, a weighted average of precision and recall (F1 score) of 93.17%, a precision of 97.27%, and a recall of 89.41%. This model showed significant improvements in all aspects compared to existing mainstream YOLO series detection models, with strong versatility and robustness. This method eliminates the influence of the background and directly detects the tea bud picking points with almost no missed detections, providing accurate two-dimensional coordinates for the tea bud picking points, with a positioning precision of 96.41%. This provides a strong theoretical basis for future tea bud picking.

show abstract

Section: Discussionmentioning

confidence: 99%

A tea bud segmentation, detection and picking point localization based on the MDY7-3PTB model

Zhang,

Sun,

Xie

et al. 2023

Front. Plant Sci.

View full text Add to dashboard Cite

show abstract

“…Although these models give good results, the need for an external reference during the conversation limits the user. In Turkish sign language [47] and in Malaysian sign language [48] suggested CNN-based methods. In addition, [49] presents a CNN-based recognition system using sensors in their study.…”

Section: Comparison With Other Methodsmentioning

confidence: 99%

Real-time sign language recognition based on YOLO algorithm

Alaftekin,

Pacal,

Cicek

2024

Neural Comput & Applic

View full text Add to dashboard Cite

This study focuses on real-time hand gesture recognition in the Turkish sign language detection system. YOLOv4-CSP based on convolutional neural network (CNN), a state-of-the-art object detection algorithm, is used to provide real-time and high-performance detection. The YOLOv4-CSP algorithm is created by adding CSPNet to the neck of the original YOLOv4 to improve network performance. A new object detection model has been proposed by optimizing the YOLOv4-CSP algorithm in order to provide more efficient detection in Turkish sign language. The model uses CSPNet throughout the network to increase the learning ability of the network. However, Proposed YOLOv4-CSP has a learning model with Mish activation function, complete intersection of union (CIoU) loss function and transformer block added. The Proposed YOLOv4-CSP algorithm has faster learning with transfer learning than previous versions. This allows the proposed YOLOv4-CSP algorithm to perform a faster restriction and recognition of static hand signals simultaneously. To evaluate the speed and detection performance of the proposed YOLOv4-CSP model, it is compared with previous YOLO series, which offers real-time detection, as well. YOLOv3, YOLOv3-SPP, YOLOv4-CSP and proposed YOLOv4-CSP models are trained with a labeled dataset consisting of numbers in Turkish Sign language, and their performances on the hand signals recognitions are compared. With the proposed method, 98.95% precision, 98.15% recall, 98.55 F1 score and 99.49% mAP results are obtained in 9.8 ms. The proposed method for detecting numbers in Turkish sign language outperforms other algorithms with both real-time performance and accurate hand sign prediction, regardless of background.

show abstract

“…4, there are four residual blocks in ResNet12 with three convolutional layers in each block. According to the model performance in the primary experiment, we decided to apply "within blocks" strategy [40] by inserting CBAM into every residual block. The exact position of CBAM is suitable for refining the intermediate feature maps obtained from the previous convolutional layers and increasing the robustness of the features against translation through the following pooling layer.…”

Section: Attention-based Feature Learnermentioning

confidence: 99%