Eyetracking in cognitive state detection for HCI

Abstract

1. Past research in a number of fields confirms the existence of a link between cognition and eye movement control, beyond simply a pointing relationship. This being the case, it should be possible to use eye movement recording as a basis for detecting users' cognitive states in real time. Several examples of such cognitive state detectors have been reported in the literature. 2.A multi-disciplinary project is described in which the goal is to provide the computer with as much real-time information about the human state (cognitive, affective and motivational state) as possible, and to base computer actions on this information. The application area in which this is being implemented is science education, learning about gears through exploration. Two studies are reported in which participants solve simple problems of pictured gear trains while their eye movements are recorded. The first study indicates that most eye movement sequences are compatible with predictions of a simple sequential cognitive model, and it is suggested that those sequences that do not fit the model may be of particular interest in the HCI context as indicating problems or alternative mental strategies. The mental rotation of gears sometimes produces sequences of short eye movements in the direction of motion; thus, such sequences may be useful as cognitive state detectors.3. The second study tested the hypothesis that participants are thinking about the object to which their eyes are directed. In this study, the display was turned off partway through the process of solving a problem, and the participants reported what they were thinking about at that time. While in most cases the participants reported cognitive activities involving the fixated object, this was not the case on a sizeable number of trials.

Measuring Cognitive Load using Eye Tracking Technology in Visual Computing

“…Previous research has shown that users' pupils dilate when the difficulty of the task and their cognitive effort to solve it increase [6,19,20,18,14]. Many studies have validated this argument across a variety of tasks, including reading, problem solving, and visual tasks [21]. Porta et al [19] also observed a decrease in pupil diameter size towards the end of the task in their experiment, interpreting it as a potential sign for tiredness.…”

“…Thus, the state of attention is indicated by the position of the gaze within the display or a specific AOI. Rudmann et al [21] found that the gaze direction indicates the interface element that is relevant for the current cognitive activity. It can be interpreted as a repeated interest in an area.…”

“…Further, Rudmann et al [21] have used the investigation of AOIs in relation to cognitive states. Their results show that users actually visually focus on relevant parts of the system when solving a task.…”

“…To this end, several studies have investigated the relation of voluntary eye-movements like fixations and saccades on cognitive processes [21] as well as involuntary eye-movements like blink rate and pupil dilation [17]. These eye movements are also called behavioural (voluntary) and physiological (involuntary) [5].…”

Added Value of Eye Tracking in Usability Studies: Expert and Non-expert Participants

“…The human eye reads a line of text in discrete chunks through a series of fixations and fast eye movements to take in the next section [13]. Eye tracking is based on the fact that a record of a person's eye movements while doing a task provides information about the nature, sequence and timing of the cognitive operations that took place [14].…”