A gaze interactive assembly instruction with pupillometric recording

Abstract

This paper presents a study of a gaze interactive digital assembly instruction that provides concurrent logging of pupil data in a realistic task setting. The instruction allows hands-free gaze dwells as a substitute for finger clicks, and supports image rotation as well as image zooming by head movements. A user study in two LEGO toy stores with 72 children showed it to be immediately usable by 64 of them. Data logging of view-times and pupil dilations was possible for 59 participants. On average, the children spent half of the time attending to the instruction (S.D. 10.9%). The recorded pupil size showed a decrease throughout the building process, except when the child had to back-step: a regression was found to be followed by a pupil dilation. The main contribution of this study is to demonstrate gaze-tracking technology capable of supporting both robust interaction and concurrent, non-intrusive recording of gaze- and pupil data in-the-wild. Previous research has found pupil dilation to be associated with changes in task effort. However, other factors like fatigue, head motion, or ambient light may also have an impact. The final section summarizes our approach to this complexity of real-task pupil data collection and makes suggestions for how future applications may utilize pupil information.

Remote Eye Gaze Tracking Research: A Comparative Evaluation on Past and Recent Progress

“…Old reflection-based methods face challenges when applied in real-world settings because they require dedicated NIR devices. These methods are majorly subject to eye occlusion, and other reflection irregularities, such as pupil dilation [19,153,154]. To solve these bottlenecks, recent attempts by researchers have been based on techniques which focus on image appearance to extract the characteristics of the entire eye region instead of specific features that require a dedicated device [155].…”

Pupillary measurement during an assembly task

“…Recent studies by Čegovnik et al (2018) and Hansen et al (2018) has found that even a low-cost 30 Hz remote gaze tracker may provide reliable pupil measures that corresponds well with changes in task load during simulated driving and assembling of construction toy, respectively.…”

Don't hide the instruction manual: A dynamic trade-off between using internal and external templates during visual search

“…Then, the external world may not only provide the challenge (e.g., the search display), but also ease the challenge, by allowing to resample the template. Indeed, during many tasks, humans look back and forth at instructions in order to help them succeed ( Hayhoe, Shrivastava, Mruczek, & Pelz, 2003 ; Droll & Hayhoe, 2007 ; Hansen, Mardanbegi, Biermann, & Bækgaard, 2018 ; Alfandari, Belopolsky, & Olivers, 2019 ; Sullivan, Ludwig, Damen, Mayol-Cuevas, & Gilchrist, 2021 ). This behavior is in line with earlier findings on mental effort and memory, which indicate that offloading memory is preferred as much as possible over storing internally, by (re)sampling from the environment in a just-in-time manner ( O’Regan, 1992 ; Hayhoe et al., 2003 ; Triesch, Ballard, Hayhoe, & Sullivan, 2003 ; Droll, Hayhoe, Triesch, & Sullivan, 2005 ; Risko & Dunn, 2015 ; Risko & Gilbert, 2016 ; Melnik, Schüler, Rothkopf, & König, 2018 ; Somai, Schut, & Van der Stigchel, 2020 ; Van der Stigchel, 2020 ; Draschkow, Kallmayer, & Nobre, 2021 ).…”