Gaze Gesture Recognition by Graph Convolutional Networks

Abstract

Gaze gestures are extensively used in the interactions with agents/computers/robots. Either remote eye tracking devices or head-mounted devices (HMDs) have the advantage of hands-free during the interaction. Previous studies have demonstrated the success of applying machine learning techniques for gaze gesture recognition. More recently, graph neural networks (GNNs) have shown great potential applications in several research areas such as image classification, action recognition, and text classification. However, GNNs are less applied in eye tracking researches. In this work, we propose a graph convolutional network (GCN)–based model for gaze gesture recognition. We train and evaluate the GCN model on the HideMyGaze! dataset. The results show that the accuracy, precision, and recall of the GCN model are 97.62%, 97.18%, and 98.46%, respectively, which are higher than the other compared conventional machine learning algorithms, the artificial neural network (ANN) and the convolutional neural network (CNN).

Boosted Gaze Gesture Recognition Using Underlying Head Orientation Sequence

“…A) COMPARISON WITH BASELINE METHOD Table 11 depicted the performance comparison of our adopted method and method Shi et al [7]. The table depict the average performance recognition of boosted HMM.…”

“…The table depict the average performance recognition of boosted HMM. The average recognition performance of the RF classifier [7] is 88.1%, 90.77%, and 85.03% for accuracy, precision, and recall respectively. However, the performance results of boosted-HMM with the same dataset are shown in orange color in figure 7.…”

“…To solve the problem of gaze gesture recognition, authors in [7] introduced GCN and utilized gaze points in 2D velocity as node. However, multicollinearity caused by these velocity nodes is more vulnerable in complex gestures because there are more subunits of eye fixations f and saccades s transitions in complex gestures.…”

“…Therefore, a suitable ML algorithm is needed to improve the recognition of complex gaze gestures in smart system applications. In Shi et al [7], a graph convolution network (GCN) was used to recognize nine simple gaze gestures from gaze points of optical flow approximations. The GCN treated each gaze point of the optical flow as a node and generate edges to connect each successive node.…”

“…Hence, an effective approach to transform and decompose these subunits is needed. To decompose subunits in complex gaze gestures, we present a new set of gaze points that are obtained from eye gaze sequence and head orientation angles, which were absent in [7]. These new sets of gaze point have more distinguishable information and small underlying orientation changes for complex gaze gestures.…”