Automatic analysis of eye-tracking data using object detection algorithms

Abstract

In this paper we investigate the integration of object detection algorithms with eye-tracking data. The emerging technology of lightweight mobile eye-trackers enables realistic in-the-wild user experience experiments. Unfortunately, mobile eye-trackers generate a large amount of video data, which up to now requires manual analysis. This time-consuming and repetitive task renders processing large datasets economically infeasible. Our main contribution is the use of object detection algorithms to perform this analysis task automatically. We compare several object detection algorithms with regard to both speed and accuracy. To prove their functionality, we have recorded an eye-tracker shopping experiment and processed the data using object detection techniques.

You-Do, I-Learn: Egocentric unsupervised discovery of objects and their modes of interaction towards video-based guidance

“…Its anticipatory nature allows estimating which object will be used next [30,29]. Gaze has been successfully used to assist action and activity recognition [12,35,44,38] and supervised object recognition [53,11].…”

“…Two consecutive image parts, Ω(I t ) and Ω(I t−1 ) belong to the same object if ||f t − f t−1 || < 1 , where 1 is the threshold selected to accept clustering consecutive image parts and ||.|| is the Euclidean distance (Algo. 1 L. [9][10][11][12][13][14]. The strict consecutive constraint between t and t−1 can be relaxed to allow proximity within a sliding window.…”

Gaze-Based Assistive Technologies

“…offline processing. They presented first results of their prototype system a year later (De Beugher, Ichiche, Brône & Goedemé, 2012). While the described approaches do not focus on providing real-time performance on a mobile device, they are facing all of the computer vision (lighting, occlusions, and fast movements) and localization problems.…”

Hybrid Eye-Tracking on a Smartphone with CNN Feature Extraction and an Infrared 3D Model

“…The capability to estimate where a subject is looking is known as gaze estimation or eye-tracking. This technology has enhanced applications in a wide array of domains, including the measurement of advertising efficacy [1,2], instrumentation to enhance reading [3][4][5], automotive safety [6,7], pilot training [8][9][10], accessibility interfaces [11][12][13][14], and provide objective indicators of cognitive, psychiatric, and neurological states of individuals [15][16][17][18][19][20][21][22][23][24][25][26]. Even though research with specialized eye-tracking systems demonstrated that such systems can be used in many domains, the need to purchase expensive, specialized software and hardware to monitor the PoG limit the use of eye-tracking systems by consumers.…”