Convolutional neural networks on eye tracking trajectories classify patients with spatial neglect

Abstract

<jats:p>Eye-movement trajectories are rich behavioral data, providing a window onto how the brain processes information. Analyses of these trajectories can be automated and benefit from machine learning algorithms. Among those, deep learning has recently proven very successful, setting new state-of-art results in many computer vision applications, including medical diagnosis systems. In this paper, we address the challenge of diagnosing and quantifying signs of visuospatial neglect from saccadic eye trajectories recorded in healthy controls and in brain-damaged patients with spatial neglect. We show how machine learning techniques, such as deep networks, can predict the patient's status with unprecedented accuracy, benchmarking the algorithm prediction with structural Magnetic Resonance Images (MRI) of the patients' brain lesions and their Diffusion Tensor Imaging (DTI) tracts. Preliminary evidence of correlation between MRI data and the algorithm scores suggest that a quantitative prediction of the patients' impairment based only onto the behavioral data of eye trajectories seem possible, therefore opening to new horizons in the field of non-invasive diagnostics.</jats:p>