Emotion Recognition Using Eye-Tracking: Taxonomy, Review and Current Challenges

Abstract

The ability to detect users’ emotions for the purpose of emotion engineering is currently one of the main endeavors of machine learning in affective computing. Among the more common approaches to emotion detection are methods that rely on electroencephalography (EEG), facial image processing and speech inflections. Although eye-tracking is fast in becoming one of the most commonly used sensor modalities in affective computing, it is still a relatively new approach for emotion detection, especially when it is used exclusively. In this survey paper, we present a review on emotion recognition using eye-tracking technology, including a brief introductory background on emotion modeling, eye-tracking devices and approaches, emotion stimulation methods, the emotional-relevant features extractable from eye-tracking data, and most importantly, a categorical summary and taxonomy of the current literature which relates to emotion recognition using eye-tracking. This review concludes with a discussion on the current open research problems and prospective future research directions that will be beneficial for expanding the body of knowledge in emotion detection using eye-tracking as the primary sensor modality.

Development and Calibration of an Eye-Tracking Fixation Identification Algorithm for Immersive Virtual Reality

“…In the world-centered design the gaze is directed at the 2D display, while the subjects is positioned in front of a screen, with restricted head movement [26]. This system is used in experimental designs where remote or desktop-integrated eye trackers are used [27]. The head-centered design, on the other hand, measures gaze points within the video recording, which moves as the subject moves his/her head.…”

“…This has big advantages in real-world experiments, but involves some difficulties in automatically identifying what the subject is looking at. This type of system derives from advanced technologies, such as mobile eye trackers (MET) [27]. However, the eye trackers integrated into HMDs present a new VR-centered framework, where the origin of the coordinate system is the virtual environment.…”

Progress and Challenges in Generative Product Design: A Review of Systems

“…In other words, where a machine can display an image and detect whether the viewer likes or dislikes it without conscious effort or action from the user. Eye-tracking has also been used for the same purpose [89]. Though less accurate for distinguishing emotions, it has the advantage that the gaze locations could be used to localize the object part that the user is reacting to.…”

Advances in Multimodal Emotion Recognition Based on Brain–Computer Interfaces

“…Concerning multimodal integrated acquisition devices, the VR-capable headset LooxidVR produced by LooxidLabs (Daejeon, Korea) integrates head-mounted display (HMD) with built-in EEG sensors and eye tracking sensors. In addition, a phone can be attached to display virtual reality (VR) content [ 109 ]. This method can achieve direct synchronization and synchronous acquisition of eye movement tracking and matching EEG data, thus realizing high-fidelity synchronized eye-movement tracking and EEG data to augment VR experiences.…”