Small eye movements cannot be reliably measured by video-based P-CR eye-trackers

Abstract

For evaluating whether an eye-tracker is suitable for measuring microsaccades, Poletti & Rucci (2016) propose that a measure called 'resolution' could be better than the more established root-mean-square of the sample-to-sample distances (RMS-S2S). Many open questions exist around the resolution measure, however. Resolution needs to be calculated using data from an artificial eye that can be turned in very small steps. Furthermore, resolution has an unclear and uninvestigated relationship to the RMS-S2S and STD (standard deviation) measures of precision (Holmqvist & Andersson, 2017, p. 159-190), and there is another metric by the same name (Clarke, Ditterich, Drüen, Schönfeld, and Steineke 2002), which instead quantifies the errors of amplitude measurements. In this paper, we present a mechanism, the Stepperbox, for rotating artificial eyes in arbitrary angles from 1 ′ (arcmin) and upward. We then use the Stepperbox to find the minimum reliably detectable rotations in 11 video-based eye-trackers (VOGs) and the Dual Purkinje Imaging (DPI) tracker. We find that resolution correlates significantly with RMS-S2S and, to a lesser extent, with STD. In addition, we find that although most eye-trackers can detect some small rotations of an artificial eye, the rotations of amplitudes up to 2 ∘ are frequently erroneously measured by video-based eye-trackers. We show evidence that the corneal reflection (CR) feature of these eye-trackers is a major cause of erroneous measurements of small rotations of artificial eyes. Our data strengthen the existing body of evidence that video-based eye-trackers produce errors that may require that we reconsider some results from research on reading, microsaccades, and vergence, where the amplitude of small eye movements have been measured with past or current video-based eye-trackers. In contrast, the DPI reports correct rotation amplitudes down to 1 ′ .

Is apparent fixational drift in eye-tracking data due to filters or eyeball rotation?

“…After a brief resting period, data recording was then started and ran for 19 s while the experimenter left the room. This procedure followed established practice in the field (e.g., Coey et al, 2012; Wang et al, 2016;Holmqvist and Blignaut, 2020) and was required because not all the eye trackers would deliver data without a prior calibration, and we did not have a way to perform a calibration using the artificial eyes themselves.…”

“…For this reason, such research is usually conducted with the video-based eye trackers that provide the lowest noise levels in their class such as the various SR-Research EyeLinks (e.g., Engbert & Mergenthaler, 2006;Scholes et al, 2015;Nyström et al, 2017) and recently the Tobii Spectrum (Nyström et al, in press). Note also that it has recently been questioned whether video-based eye trackers are suitable for microsaccade research (Holmqvist & Blignaut, 2020). Other researchers interested in fixational eye movements use different measurement techniques that provide even better data quality (lower measurement noise levels), such as Dual-Purkinje eye trackers (Kuang et al, 2012;Horowitz et al, 2007), scleral search coils (McCamy et al, 2015;Ko et al, 2016) and various scanning laser opthalmoscopes (Stevenson et al, 2010;Sheehy et al, 2012;Bowers et al, 2019).…”

The mean point of vergence is biased under projection

“…• data from eye-trackers have an inaccuracy (systematic error) and introduce imprecision (variable error, or noise) onto the signal (Holmqvist et al, 2011;Wang et al, 2017a;Niehorster et al, 2018). • the inaccuracy is not constant, but varies with pupil size (Drewes, 2014) and quantization of the CR in the eye camera (Holmqvist et al, 2019). • ideally, human gaze direction is controlled to bring the object into the fovea centrals (Atchison and Smith, 2000), which has a non-negligible extent of 1.5−2°.…”

“…However, even if the observer experiences looking at a point in space with both eyes, the eye rays provided by the eye-tracker contain error, for a variety of reasons: data from eye-trackers have an inaccuracy (systematic error) and introduce imprecision (variable error, or noise) onto the signal [ 15 , 16 , 17 ]. the inaccuracy is not constant, but varies with pupil size [ 18 ] and quantization of the CR in the eye camera [ 19 ]. ideally, human gaze direction is controlled to bring the object into the fovea centrals [ 20 ], which has a non-negligible extent of 1.5–2°.…”

“…the inaccuracy is not constant, but varies with pupil size [ 18 ] and quantization of the CR in the eye camera [ 19 ].…”