Scanpath comparisons for complex visual search in a naturalistic environment

Abstract

Naturalistic surveillance tasks provide a rich source of eye-tracking data. It can be challenging to make meaningful comparisons using standard eye-tracking analysis techniques such as saccade frequency or blink rate in surveillance studies due to the temporal irregularity of events of interest. Naturalistic research environments present unique challenges, such as requiring specialized or expert analysts, small sample size, and long data collection sessions. These constraints demand rich data and sophisticated analyses, particularly in prescriptive naturalistic environments where problems must be thoroughly understood to implement effective and practical solutions. Using a small sample of expert surveillance analysts and an equalsized sample of novices, we computed scanpath similarity on a variety of surveillance data using the ScanMatch Matlab tool. ScanMatch implements an algorithm initially developed for DNA protein sequence comparisons and provides a similarity score for two scanpaths based on their morphology and, optionally, duration in an area of interest. Both experts and novices showed equal dwell time on targets regardless of identification accuracy and both samples showed higher scanpath consistency across participants as a function of target type rather than individual subjects showing a particular scanpath preference. Our results show that scanpath analysis can be leveraged as a highly effective computer-based methodology to characterize surveillance identification errors and guide the implementation of solutions. Similarity scores can also provide insight into processes guiding visual search. Keywords Visual search • Scanpath analysis • Applied research Background Variables and Data Collection Tools in Surveillance Research Naturalistic research provides many opportunities to understand cognitive phenomena in real-life working environments. By examining cognition as it naturally unfolds, it becomes easier to develop a fuller understanding of applied research problems and implement reasonable solutions, but there are challenges that are not typical in laboratory studies. Naturalistic environments require laboratory tasks that are high fidelity to the environment where software and Mary E. Frame

Dynamic scan paths investigations under manual and highly automated driving

“…Thus, in future investigations, knowledge or methodologies might lead to reconsider the proposed categories. Alternative methodologies to devise scan paths clusters could be foreseen, based on automatic clustering methods for instance 70 . To facilitate that process, all the scan paths combinations and relatives' frequencies were made available (Tables 1, 2, 3).…”

Eye-movement replay supports episodic remembering

“…To measure scanpath similarity between encoding and recall, we used MultiMatch (MM), which was introduced as a method for comparing scanpaths over different spatiotemporal dimensions 37 , and has subsequently been validated, applied, and evaluated against other scanpath comparison tools 35,[38][39][40][41] . The basic principle is that the MM algorithm simplifies the two scanpaths under comparison (in this case, one from encoding and the other from recall) into virtualized sequences of saccadic vectors and then temporally aligns them with each other.…”

Impact of video content and resolution on the cognitive dynamics of surveillance decision‐making

“…Thus, it is essential to et al, 2017). Eye tracking is one example of a process tracing method that has been leveraged in visual search to understand how attention shifts and encoding occurs during a search task (Frame, Warren, & Maresca, 2019;Martin et al, 2018). Although much research links eye tracking variables to visual search, eye tracking equipment is not always affordable or possible to implement in applied settings.…”

“…Process tracing methods refer to a class of methodologies that can capture sophisticated information about attention, information integration and weighting, preference reversals, and other cognitive features that comprise a process such as categorization or decision‐making (Schulte‐Mecklenbeck et al, 2017). Eye tracking is one example of a process tracing method that has been leveraged in visual search to understand how attention shifts and encoding occurs during a search task (Frame, Warren, & Maresca, 2019; Martin et al, 2018). Although much research links eye tracking variables to visual search, eye tracking equipment is not always affordable or possible to implement in applied settings.…”