DeepVOG: Open-source pupil segmentation and gaze estimation in neuroscience using deep learning

Yiu, Yuk-Hoi; Aboulatta, Moustafa; Raiser, Theresa; Ophey, Leoni; Flanagin, Virginia L.; Eulenburg, Peter zu; Ahmadi, Seyed-Ahmad

doi:10.1016/j.jneumeth.2019.05.016

Cited by 131 publications

(76 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Use pupil size and point-of-gaze for predicting the users' behaviours (e.g., word searching, question answering, looking for the most interesting title in a list) [28] Naïve Bayes classifier Use of fixation duration, mean, and standard deviation to identify various visual activities (e.g., reading, scene search) [29] MLP Use of pupil dilation, gaze dispersion to classify various tasks on decision making [30] Decision tree, MLP, support vector machines (SVM), linear regression Use of fixation rate, fixation duration, fixations per trial, saccade amplitude, and relative saccade angles to identify eye movements to predict visualisation tasks In addition to conventional methods, existing works also utilise the deep learning (DL) approaches for the pupil detection while using hierarchical image patterns to enhance and eliminate artefacts with Convolutional Neural Networks (CNNs). For instance, [21] proposed the use of fully connected CNNs for segmentation of the entire pupil area in which they trained the network on 3946 video oscillography images. These images were hand annotated and generated within a laboratory environment.…”

Section: Referencementioning

confidence: 99%

“…Despite the variety of existing methods for the pupil localisation, further improvements are required in terms of a precise estimation for the pupil location. For instance, the DL-based pupil localisation and gaze estimation in [21] uses pixel distance to validate the performance, which is not a standard representation of the error in the case of varying resolutions. Furthermore, the validation is performed on a dataset containing artificially rendered images, which in most cases do not reflect the real-time dynamics.…”

Section: Referencementioning

confidence: 99%

“…Standard methods in gaze estimation are based on corneal reflections that needs an accurate localisation of the pupil centre as well as the glints [21]. Pupil and glints localisation algorithms are usually based on image processing such as morphological operators for the detection of contour [22] and intensity threshold identification followed by the fitting using ray-based ellipse [23].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Pupil Localisation and Eye Centre Estimation Using Machine Learning and Computer Vision

Khan

Hussain

Kuru

et al. 2020

Sensors

View full text Add to dashboard Cite

Various methods have been used to estimate the pupil location within an image or a real-time video frame in many fields. However, these methods lack the performance specifically in low-resolution images and varying background conditions. We propose a coarse-to-fine pupil localisation method using a composite of machine learning and image processing algorithms. First, a pre-trained model is employed for the facial landmark identification to extract the desired eye frames within the input image. Then, we use multi-stage convolution to find the optimal horizontal and vertical coordinates of the pupil within the identified eye frames. For this purpose, we define an adaptive kernel to deal with the varying resolution and size of input images. Furthermore, a dynamic threshold is calculated recursively for reliable identification of the best-matched candidate. We evaluated our method using various statistical and standard metrics along with a standardised distance metric that we introduce for the first time in this study. The proposed method outperforms previous works in terms of accuracy and reliability when benchmarked on multiple standard datasets. The work has diverse artificial intelligence and industrial applications including human computer interfaces, emotion recognition, psychological profiling, healthcare, and automated deception detection.

show abstract

Section: Referencementioning

confidence: 99%

Section: Referencementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Pupil Localisation and Eye Centre Estimation Using Machine Learning and Computer Vision

Khan

Hussain

Kuru

et al. 2020

Sensors

View full text Add to dashboard Cite

show abstract

“…Additionally, an integrated video documentation and analysis system will analyse eye movements and gait abnormalities by the means of pattern recognition (e.g. neuronal networks, deep learning algorithms) [20][21][22]. It will be used to objectify the diagnosis and classify patient's self-recorded symptoms.…”

Section: Interventionmentioning

confidence: 99%

Computerized clinical decision system and mobile application with expert support to optimize management of vertigo in primary care: study protocol for a pragmatic cluster-randomized controlled trial

et al. 2020

View full text Add to dashboard Cite

Vertigo and dizziness are amongst the most common symptoms in medicine and often have a major impact on activities of daily life. Although many causes of vertigo and dizziness can easily be recognized, patients often receive inappropriate and ineffective treatment. The reasons for this are various. Because vertigo/dizziness is an interdisciplinary symptom and there is a lack of standardised diagnostic tools, it is easy to lose the overview of the possible differential diagnoses. There is evidence though, that the management of patients with vertigo/dizziness can be optimized using standardized care pathways with digital support. The present study (within the framework of "PoiSe-prevention, online feedback, and interdisciplinary therapy of acute vestibular syndromes by e-health") aims to evaluate the implementation of a program with several interlocking components. The three main components are a computerized clinical decision system, a mobile application, a counselling and interdisciplinary educational program developed by the German Center for Vertigo and Balance Disorders (DSGZ). The study is a cluster-randomized controlled trial with a parallel-group design, as well as a detailed process evaluation. Clusters comprise of primary care physician practices in Bavaria, Germany. In the scope of the study the effectiveness, acceptability and efficiency of the intervention will be evaluated. It is anticipated that the intervention will improve the quality and efficiency of the management of dizzy patients. A higher diagnostic accuracy, optimized treatment, and disease progression monitoring is expected to improve patient-relevant outcomes and reduce health-care costs.

show abstract

“…Other eye trackers which are based on high speed video cameras are more expensive than the infrared based eye trackers, but are also more accurate than webcam based eye trackers (Agarwal et al, 2019a). In such eye trackers, the measurement is done based on deep learning and computer vision applications (Kato et al, 2019), (Yiu et al, 2019).…”

Section: Introductionmentioning

confidence: 99%

Eye Gaze Tracking for Detecting Non-verbal Communication in Meeting Environments

Dhingra

Hirt

Angst

et al. 2020

Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Application

View full text Add to dashboard Cite

Non-verbal communication in a team meeting is important to understand the essence of the conversation. Among other gestures, eye gaze shows the focus of interest on a common workspace and can also be used for an interpersonal synchronisation. If this non-verbal information is missing and or cannot be perceived by blind and visually impaired people (BVIP), they would lack important information to get fully immersed in the meeting and may feel alienated in the course of the discussion. Thus, this paper proposes an automatic system to track where a sighted person is gazing at. We use the open source software 'OpenFace' and develop it as an eye tracker by using a support vector regressor to make it work similarly to commercially available expensive eye trackers. We calibrate OpenFace using a desktop screen with a 2 × 3 box matrix and conduct a user study with 28 users on a big screen (161.7 cm x 99.8 cm x 11.5 cm) with a 1 × 5 box matrix. In this user study, we compare the results of our developed algorithm for OpenFace to an SMI RED 250 eye tracker. The results showed that our work achieved an overall relative accuracy of 58.54%.

show abstract

DeepVOG: Open-source pupil segmentation and gaze estimation in neuroscience using deep learning

Cited by 131 publications

References 21 publications

Pupil Localisation and Eye Centre Estimation Using Machine Learning and Computer Vision

Pupil Localisation and Eye Centre Estimation Using Machine Learning and Computer Vision

Computerized clinical decision system and mobile application with expert support to optimize management of vertigo in primary care: study protocol for a pragmatic cluster-randomized controlled trial

Eye Gaze Tracking for Detecting Non-verbal Communication in Meeting Environments

Contact Info

Product

Resources

About