Gaze Estimation via Strip Pooling and Multi-Criss-Cross Attention Networks

Abstract

Deep learning techniques for gaze estimation usually determine gaze direction directly from images of the face. These algorithms achieve good performance because face images contain more feature information than eye images. However, these image classes contain a substantial amount of redundant information that may interfere with gaze prediction and may represent a bottleneck for performance improvement. To address these issues, we model long-distance dependencies between the eyes via Strip Pooling and Multi-Criss-Cross Attention Networks (SPMCCA-Net), which consist of two newly designed network modules. One module is represented by a feature enhancement bottleneck block based on fringe pooling. By incorporating strip pooling, this residual module not only enlarges its receptive fields to capture long-distance dependence between the eyes but also increases weights on important features and reduces the interference of redundant information unrelated to gaze. The other module is a multi-criss-cross attention network. This module exploits a cross-attention mechanism to further enhance long-range dependence between the eyes by incorporating the distribution of eye-gaze features and providing more gaze cues for improving estimation accuracy. Network training relies on the multi-loss function, combined with smooth L1 loss and cross entropy loss. This approach speeds up training convergence while increasing gaze estimation precision. Extensive experiments demonstrate that SPMCCA-Net outperforms several state-of-the-art methods, achieving mean angular error values of 10.13° on the Gaze360 dataset and 6.61° on the RT-gene dataset.

From Face to Gait: Weakly-Supervised Learning of Gender Information from Walking Patterns

“…Each branch includes a fully connected layer that generates continuous as well as discrete predictions. Yan et al [27] improve on this architecture by modifying the backbone to employ strip pooling which makes the receptive field more suited to gaze estimation. They also incorporate multi-criss-cross attention to capture dependencies between the eye features.…”

“…L2CS-Net [1] uses a ResNet-50 as backbone and employs 2 different prediction heads for the yaw and pitch angles. SPMCCA-Net [27] integrates strip pooling and criss-cross attention to the ResNet backbone. DAM [12] predicts the head pose and extracts the features of the eye crops to make the gaze prediction, however it benefits from additional annotations.…”