Automating areas of interest analysis in mobile eye tracking experiments based on machine learning

Abstract

For an in-depth, AOI-based analysis of mobile eye tracking data, a preceding gaze assignment step is inevitable. Current solutions such as manual gaze mapping or marker-based approaches are tedious and not suitable for applications manipulating tangible objects. This makes mobile eye tracking studies with several hours of recording difficult to analyse quantitatively. We introduce a new machine learning-based algorithm, the computational Gaze-Object Mapping (cGOM), that automatically maps gaze data onto respective AOIs. cGOM extends state-of-the-art object detection and segmentation by mask R-CNN with a gaze mapping feature. The new algorithm’s performance is validated against a manual fixation-by-fixation mapping, which is considered as ground truth, in terms of true positive rate (TPR), true negative rate (TNR) and efficiency. Using only 72 training images with 264 labelled object representations, cGOM is able to reach a TPR of approx. 80% and a TNR of 85% compared to the manual mapping. The break-even point is reached at 2 hours of eye tracking recording for the total procedure, respectively 1 hour considering human working time only. Together with a real-time capability of the mapping process after completed training, even hours of eye tracking recording can be evaluated efficiently. (Code and video examples have been made available at: https://gitlab.ethz.ch/pdz/cgom.git)

Object-gaze distance: Quantifying near-peripheral gaze behavior in real-world applications

“…Therefore, increasing the AOI size around objects of interest can result in areas that exceed the size of the actual objects by several orders of magnitude. In our study, we assume that the most accurate depiction of an operator’s gaze behavior can only be achieved using a close contour AOI production method, such as Mask R-CNN and cGOM ( 35 ), that continuously adjusts AOI sizes to the actual object size within the dynamic scene. One advantage of our presented method is that it can be equally used for the analysis of AOI Hits, if traditional metrics are of interest to the researchers.…”

“…The semantic mapping of fixations for the AOI Hit method was implemented using the automated AOI mapping algorithm cGOM ( 35 ). The algorithm detects and segments pre-trained objects, here the screw and the screwdriver, using the Mask R-CNN network ( 14 ) and, subsequently, determines whether the gaze of each fixation lies within the constraints of the 2D pixel (px) coordinate matrix of the segmented object masks.…”

“…Recent applications of deep convolutional neural networks have used image segmentation to detect objects in mobile ET recordings and provided a way to automate the mapping of the gaze onto these detected AOIs ( 35 ). The automated gaze-object mapping algorithm processes each fixation as a pair of x- and y coordinates.…”

Automatic Visual Attention Detection for Mobile Eye Tracking Using Pre-Trained Computer Vision Models and Human Gaze

“…Previous methods tackled the automatic analysis of head-mounted eye tracking data in uninstrumented environments [ 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 ]. Drawbacks of these methods include, e.g., a missing support for real-time applications, and the restriction to a limited number of classes (≤12).…”

Introducing Point-of-Interest as an alternative to Area-of-Interest for fixation duration analysis

“…Although some researchers have suggested guidelines in defining AOIs [7,[16][17][18], there is no gold standard for defining AOIs. In addition, although methods that automatically generate AOIs have been put forward [10,[19][20][21][22][23], the dominant approach in eye-tracking studies is to manually define AOIs. Therefore, researchers often make subjective decisions in defining AOIs, causing locations, shapes, and sizes of AOIs to vary even between studies that utilize similar stimuli [10,24].…”