Ultra-low-cost 3D gaze estimation: an intuitive high information throughput compliment to direct brain–machine interfaces

Abstract

Eye movements are highly correlated with motor intentions and are often retained by patients with serious motor deficiencies. Despite this, eye tracking is not widely used as control interface for movement in impaired patients due to poor signal interpretation and lack of control flexibility. We propose that tracking the gaze position in 3D rather than 2D provides a considerably richer signal for human machine interfaces by allowing direct interaction with the environment rather than via computer displays. We demonstrate here that by using mass-produced video-game hardware, it is possible to produce an ultra-low-cost binocular eye-tracker with comparable performance to commercial systems, yet 800 times cheaper. Our head-mounted system has 30 USD material costs and operates at over 120 Hz sampling rate with a 0.5-1 degree of visual angle resolution. We perform 2D and 3D gaze estimation, controlling a real-time volumetric cursor essential for driving complex user interfaces. Our approach yields an information throughput of 43 bits s(-1), more than ten times that of invasive and semi-invasive brain-machine interfaces (BMIs) that are vastly more expensive. Unlike many BMIs our system yields effective real-time closed loop control of devices (10 ms latency), after just ten minutes of training, which we demonstrate through a novel BMI benchmark--the control of the video arcade game 'Pong'.

Neurocognitive barriers to the embodiment of technology

“…This would require the low-latency control of artificial limb movements (< 70 ms time delay), placing severe constraints on many, but not all, of the current interfacing technologies. Indeed, latencies for eye-tracking-based intention decoding are under 10 ms, which has enabled the high-precision control of robotic actuators in 3D with short training times 11 (Fig. 2e).…”

3D gaze cursor: Continuous calibration and end-point grasp control of robotic actuators

“…b) GT3D binocular eye-tracker: The GT3D binocular eye-tracker utilised in our system was developed by Abbott & Faisal, [3]. The eye-tracker is built from two commercial Playstation 3 cameras yielding a resolution of 320 x 240 pixels at a maximum frame-rate of 120Hz.…”

“…of Bioengineering & 2 Dept. of Computing, Imperial College London, South Kensington Campus, SW7 2AZ, London, UK, 3 MRC Clinical Sciences Centre. Address for correspondence: aldo.faisal@imperial.ac.uk require considerable setup and training time -even for EEG base systems.…”

“…Several different approaches have been attempted to circumvent these problems. In the recent years, eye-tracking systems have been suggested as a reliable non-invasive alternative to BMI and have been extensively developed as a cheaper and more compact interface between humans and robots [3]. The idea behind the use of eye movements as a control signal relies on the fact that our gaze is proactive and directly correlated to action intentions and cognitive control [4], [5].…”

“…Efforts have been made focusing on the enrichment of the signal interpretation and cheapening of eye-tracking devices. We have utilised an ultra low-cost eye-tracker "GT3D" custom built by the integration of two commercial Playstation3 cameras (10USD each) and two Infrared (IR) LEDs that yield pupil detection at high speed frame rates (up to 120 Hz) developed by us [3]. We have resolved the Midas touch problem here by using " Winks" for opening/closing of the gripper and "long fixations" to trigger movement to the gaze-end point -effectively proactively moving the arm to objects we examine for a long time.…”