Can Your Eyes Tell Me How You Think? A Gaze Directed Estimation of the Mental Activity

Florea, Laura; Florea, Corneliu; Vranceanu, Ruxandra; Vertan, Constantin

doi:10.5244/c.27.60

Cited by 28 publications

(9 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Voigt [13] used the Caffe reference model of Alex-Net and obtained the test accuracy of 79.27% for the CAVE-DB. George et al [14] used a CNN model with three convolution stages and seven eye gaze directions and achieved the recognition rate of 86.81% that these results were obtained for all seven of the different eye gaze directions in the Eye Chimera database [2]. Mitsuzuka et al proposed using a CNN model combined with random forests, where eye image features such as intensity, histogram of oriented gradients (HOG), and output of the CNN are collected and learned for classification [15], [16].…”

Section: Eye Gaze Direction Classification By Deep Learningmentioning

confidence: 95%

Image Augmentation for Eye Contact Detection Based on Combination of Pre-trained Alex-Net CNN and SVM

Omori¹

2020

JCP

View full text Add to dashboard Cite

Making eye contact is the most powerful mode of establishing a communicative link between humans. We propose a method for detecting eye contact (mutual gaze) from images of both eyes through the combined usage of a pre-trained convolutional neural network (CNN) and a support vector machine (SVM). Neural networks are a powerful technology for classifying object images. When it comes to classification accuracy, a huge number of training samples is the key to success. The training samples are augmented by image perturbation, namely, shifting the cropping regions. A pre-trained CNN, Alex-Net, is used as the image feature extractor after being pre-trained for large-scale object image datasets. An SVM is used as the trainable classifier. Original both-eyes samples of two classes on the Columbia Gaze Data Set CAVE-DB are divided in five-fold cross-validation. Manually cropped images and automatically augmented images on the CAVE-DB are trained by the SVM. The feature vectors of the eye images are then passed to the SVM from Alex-Net. We performed 5-fold t-testing on 77 images and found that the average error rate was 16.44%, and the lowest error rate of images without glasses was 8.96% with 7,850 training images of perturbation. These results demonstrate that the proposed method is effective in detecting eye contact.

show abstract

Section: Eye Gaze Direction Classification By Deep Learningmentioning

confidence: 95%

Image Augmentation for Eye Contact Detection Based on Combination of Pre-trained Alex-Net CNN and SVM

Omori¹

2020

JCP

View full text Add to dashboard Cite

show abstract

“…j. Some of the gaze estimation models proposed in the literature were trained on relatively small datasets [108]; hence they did not produce competitive results. Generally, well-balanced, and large-scale datasets are required to construct effective and robust deep learning models.…”

Section: A Limitation and Future Work Directionsmentioning

confidence: 99%

Convolutional Neural Network-Based Methods for Eye Gaze Estimation: A Survey

Akinyelu

Blignaut

2020

IEEE Access

View full text Add to dashboard Cite

Eye tracking is becoming a very important tool across many domains, including humancomputer-interaction, psychology, computer vision, and medical diagnosis. Different methods have been used to tackle eye tracking, however, some of them are inaccurate under real-world conditions, while some require explicit user calibration which can be burdensome. Some of these methods suffer from poor image quality and variable light conditions. The recent success and prevalence of deep learning have greatly improved the performance of eye-tracking. The availability of large-scale datasets has further improved the performance of deep learning-based methods. This paper presents a survey of the current state-of-the-art on deep learning-based gaze estimation techniques, with a focus on Convolutional Neural Networks (CNN). This paper also provides a survey on other machine learning-based gaze estimation techniques. This study aims to empower the research community with valuable and useful insights that can enhance the design and development of improved and efficient deep learning-based eye-tracking models. This study also provides information on various pre-trained models, network architectures, and open-source datasets that are useful for training deep learning models.

show abstract

“…Convolutional Neural Network pertama kali dikembangkan oleh seorang peneliti bernama Kunihiko Fukushima dari NHK Broadcasting Science Research Laboratories, Kinuta, Setagaya, Tokyo, Jepang dengan nama NeoCognitron [2]. Convolution Neural Network bermula dari Yann LeCun dan teman-temannya berhasil melakukan klasifikasi citra kode zip menggunakan kasus khusus dari Feed Forward Neural Network Pada tahun 1989 [3].…”

Section: Pendahuluanunclassified

Estimasi Arah Tatapan Mata Menggunakan Ensemble Convolutional Neural Network

Sugiarto

Kristian

Setyaningsih

2018

teknika

View full text Add to dashboard Cite

Studi arah tatapan mata adalah salah satu masalah dalam bidang computer vision. Pengetahuan akan arah tatapan mata dapat memberikan informasi berharga yang dapat dimanfaatkan untuk berbagai macam keperluan dalam bidang lainnya, khususnya dalam bidang interaksi manusia dengan komputer. Dalam paper ini nantinya akan meneliti arah tatapan mata menggunakan Ensemble Convolutional Neural Network dengan menggunakan dataset CAVE (Columbia Gaze Dataset). Convolutional Neural Netwok (CNN) merupakan sebuah bidang keilmuan dalam bidang machine learning yang berkembang cukup pesat khususnya untuk mengklasifikasi citra. Nantinya, paper ini akan menganalisa dan membandingkan hasil F1 score dan weighted kappa (w-kappa) score serta error dari klasifikasi dengan menggunakan 3, 9, dan 21 kelas. Dengan sama-sama menggunakan kanal RGB sebagai gambar input, maka dapat dibandingkan dan disimpulkan bahwa dengan menggunakan metode Ensemble Convolutional Neural Network dengan koefisien 1 untuk mata kiri, 1 untuk mata kanan, dan 3 untuk kedua mata untuk klasifikasi dengan 3 dan 9 kelas, serta dengan koefisien 1 untuk mata kiri, 1 untuk mata kanan, dan 5 untuk kedua mata untuk klasifikasi dengan 21 kelas dapat menghasilkan hasil F1 score dan w-kappa yang lebih baik, serta tingkat error yang lebih rendah daripada menggunakan koefisien dengan nilai lainnya.

show abstract

Can Your Eyes Tell Me How You Think? A Gaze Directed Estimation of the Mental Activity

Cited by 28 publications

References 35 publications

Image Augmentation for Eye Contact Detection Based on Combination of Pre-trained Alex-Net CNN and SVM

Image Augmentation for Eye Contact Detection Based on Combination of Pre-trained Alex-Net CNN and SVM

Convolutional Neural Network-Based Methods for Eye Gaze Estimation: A Survey

Estimasi Arah Tatapan Mata Menggunakan Ensemble Convolutional Neural Network

Contact Info

Product

Resources

About