Gaze estimation driven solution for interacting children with ASD

Abstract

This paper investigates gaze estimation solutions for interacting children with Autism Spectrum Disorders (ASD). Previous research shows that satisfactory accuracy of gaze estimation can be achieved in constrained settings. However, most of the existing methods can not deal with large head movement (LHM) that frequently happens when interacting with children with ASD scenarios. We propose a gaze estimation method aiming at dealing with large head movement and achieving real time performance. An intervention table equipped with multiple sensors is designed to capture images with LHM. Firstly, reliable facial features and head poses are tracked using supervised decent method. Secondly, a convolution based integer-differential eye localization approach is used to locate the eye center efficiently and accurately. Thirdly, a rotation invariant gaze estimation model is built based on the located facial features, eye center, head pose and the depth data captured from Kinect. Finally, a multi-sensor fusion strategy is proposed to adaptively select the optimal camera to estimate the gaze as well as to fuse the depth information of the Kinect with the web camera. Experimental results showed that the gaze estimation method can achieve acceptable accuracy even in LHM situation and could potentially be applied in therapy for children with ASD.

How to Build a Supervised Autonomous System for Robot-Enhanced Therapy for Children with Autism Spectrum Disorder

“…Although the designed multi-sensor system can successfully capture the child's face with large head movement, it is also a challenge to determine which camera can obtain the best view of the frontal face. To remedy this, we have proposed a multisensor selection strategy to adaptively select the optimal camera, see [44]. In the proposed strategy, all sensors are calibrated and used to capture the sensory data in parallel.…”

Two-eye model-based gaze estimation from a Kinect sensor

“…Since we estimate the final gaze using estimations of both eyes, large estimation error of any one eye caused by the large head movement will result in a low accuracy of the final estimated gaze. To remedy this, multi-view gaze estimation [19] will be investigated in our future work. Moreover, we will focus on incorporating the human's gaze with visual tracking methods [20] to recognize human's activity in human-computer or humanhuman interaction.…”

Sensing-Enhanced Therapy System for Assessing Children With Autism Spectrum Disorders: A Feasibility Study

“…Once the facial points are located, the Pose from Orthography and Scaling with ITerations (POSIT) proposed by [32] is used to calculate the head pose. To handle the large head movement challenge, we have proposed a real-time gaze estimation method by constructing a multi-sensory fusion system [33]. For those facial points that the camera and Kinect 1 can both capture, it is feasible to find their global 3D coordinates.…”