Attend and Guide (AG-Net): A Keypoints-Driven Attention-Based Deep Network for Image Recognition

Abstract

This paper presents a novel keypoints-based attention mechanism for visual recognition in still images. Deep Convolutional Neural Networks (CNNs) for recognizing images with distinctive classes have shown great success, but their performance in discriminating fine-grained changes is not at the same level. We address this by proposing an end-to-end CNN model, which learns meaningful features linking fine-grained changes using our novel attention mechanism. It captures the spatial structures in images by identifying semantic regions (SRs) and their spatial distributions, and is proved to be the key to modelling subtle changes in images. We automatically identify these SRs by grouping the detected keypoints in a given image. The "usefulness" of these SRs for image recognition is measured using our innovative attentional mechanism focusing on parts of the image that are most relevant to a given task. This framework applies to traditional and fine-grained image recognition tasks and does not require manually annotated regions (e.g. boundingbox of body parts, objects, etc.) for learning and prediction. Moreover, the proposed keypoints-driven attention mechanism can be easily integrated into the existing CNN models. The framework is evaluated on six diverse benchmark datasets. The model outperforms the state-of-the-art approaches by a considerable margin using Distracted Driver V1 (Acc: 3.39%), Distracted Driver V2 (Acc: 6.58%), Stanford-40 Actions (mAP: 2.15%), People Playing Musical Instruments (mAP: 16.05%), Food-101 (Acc: 6.30%) and Caltech-256 (Acc: 2.59%) datasets.

Physical Education Teaching Strategy under Internet of Things Data Computing Intelligence Analysis

“…The computer vision-based sports action recognition systems can provide rapid postmatch analysis and real-time objective feedback before the next race for coaches and players. Bera et al [ 12 ] pointed out that the fundamental points of athletes' actions could be captured via three-dimensional video shooting techniques. Yu and Chin [ 13 ] recorded athletes' time-spatial action images via radio frequency technology in IoT, which turned out to be blurred and could not be seen clearly.…”

Driver Anomaly Quantification for Intelligent Vehicles: A Contrastive Learning Approach With Representation Clustering

“…[9] presented an ensemble of four CNN models to handle different parts of the driver, including the face, hands, and body, to recognize driver activity. [38] proposed an attend and guide network to classify driver behavior by obtaining the spatial structures of images through the identification of semantic regions and their spatial distributions. [39] concatenated three CNN models to construct a hybrid framework for detecting distracted driver behavior.…”

SR-GNN: Spatial Relation-Aware Graph Neural Network for Fine-Grained Image Categorization

“…Moreover, partbased methods limit both scalability and practicality of realworld FGVC applications. Thus, many recent methods have used image-level labels to guide their models in identifying the key object parts to discriminate the sub-categories by exploring attention mechanisms in the image space or feature space [7]- [10] to automatically mine discriminative features.…”