A Computational Model of “Active Vision” for Visual Search in Human–Computer Interaction

Abstract

Human visual search plays an important role in many human-computer interaction (HCI) tasks. Better models of visual search are needed not just to predict overall performance outcomes, such as whether people will be able to find the information needed to complete an HCI task, but to understand the many human processes that interact in visual search, which will in turn inform the detailed design of better user interfaces. This article describes a detailed instantiation, in the form of a computational cognitive model, of a comprehensive theory of human visual processing known as ''active vision'' (Findlay & Gilchrist, 2003). The computational model is built using the Executive Process-Interactive Control cognitive architecture. Eye-tracking data from three experiments inform the development and validation of the model. The modeling asks-and at least partially answers-the four questions of active vision: (a) What can be perceived in a fixation? (b) When do the eyes move? (c) Where do the eyes move? (d) What information is integrated between eye movements? Answers include: (a) Items nearer the point of gaze are more likely to be perceived, and the visual features of objects are sometimes misidentified. (b) The eyes move after the fixated visual stimulus has been processed (i.e., has entered working memory). (c) The eyes tend to go to nearby objects. (d) Only the coarse spatial information of what has been fixated is likely maintained between fixations. The model developed to answer these questions has both scientific and practical value in that the model gives Tim Halverson is a cognitive scientist with an interest in human-computer interaction, cognitive modeling, eye movements, and fatigue; he is a Research Computer Scientist in the Applied Neuroscience Branch of the Air Force Research Laboratory. Anthony Hornof is a computer scientist with an interest in human-computer interaction, cognitive modeling, visual search, and eye tracking; he is an Associate Professor in the

Understanding multitasking through parallelized strategy exploration and individualized cognitive modeling

“…Paper-based cognitive modeling [3] has evolved into computational cognitive modeling [11] which permits increased complexity and predictive power in the models. Eye tracking has emerged as a powerful technique for revealing in greater detail how people are truly interacting with devices [6] and can also reveal nuanced interactions between parallelized task hierarchies [8].…”

A concurrent real-time biologically-inspired visual object recognition system

“…with D i = {x|∀x ∈ D i : ∄y ∈ D j , i = j : r(x, y) > θ} (13) with θ being a threshold of the response of our approximated radial basis function r of Equation 11. PseudoAlgorithm 1 displays how a sub-dictionary is created.…”

“…Some of these models have also been applied to enhance common techniques like face recognition by using biologically-inspired features [6]. Some research draw more attention to active-vision systems, which have been used to solve different vision problems like: object recognition [7], [8], [9], [10], [11]; visual search [12], [13]; visual attention [14]; or visual tracking [15]. It has also been investigated how to integrate object recognition [16], [17] and visual attention also with a focus on the aspect of computational complexity [18].…”

Interaction science SIG

“…Theories often consist of some systematic representation of individual and or collective interaction that leads to the generation of testable hypotheses. For example, a scientific theory might consist of a computational or mathematical specification of the human information processing mechanisms [11,4], for others the science is more in the empirical method [8,18,20]. Cumulative theory offers explanations across a broad range of contexts and phenomena.…”