Image Augmentation for Eye Contact Detection Based on Combination of Pre-trained Alex-Net CNN and SVM

Abstract

Making eye contact is the most powerful mode of establishing a communicative link between humans. We propose a method for detecting eye contact (mutual gaze) from images of both eyes through the combined usage of a pre-trained convolutional neural network (CNN) and a support vector machine (SVM). Neural networks are a powerful technology for classifying object images. When it comes to classification accuracy, a huge number of training samples is the key to success. The training samples are augmented by image perturbation, namely, shifting the cropping regions. A pre-trained CNN, Alex-Net, is used as the image feature extractor after being pre-trained for large-scale object image datasets. An SVM is used as the trainable classifier. Original both-eyes samples of two classes on the Columbia Gaze Data Set CAVE-DB are divided in five-fold cross-validation. Manually cropped images and automatically augmented images on the CAVE-DB are trained by the SVM. The feature vectors of the eye images are then passed to the SVM from Alex-Net. We performed 5-fold t-testing on 77 images and found that the average error rate was 16.44%, and the lowest error rate of images without glasses was 8.96% with 7,850 training images of perturbation. These results demonstrate that the proposed method is effective in detecting eye contact.

Gaze estimation using convolutional neural networks

“…The findings of different eye tracking detection methods, as well as our suggested method, are summarized in Table 5. Chen and Shi [10] proposed Dilated-Net and used it to quantitatively analyze utilizing the Columbia Gaze [17] and MPIIGaze's [18] datasets high-resolution features, achieving detection accuracy of 62%.Omori and Shima [14] proposed combining a pre-trained convolutional neural network (CNN) and a Support Vector Machine to detect eye contact (mutual gaze) from photographs of both eyes (SVM). Their detection accuracy of 91.04% without glasses is up 5% from the best accuracy of 86% when using glasses.…”

“…In 2020, Omori and Shima [14] suggested an eye contact detection approach bycombinga pre-trained CNN and SVM. Furthermore, they demonstrated through tests that pre-trained CNN for object image datasets may be used as a feature extractor for both eye region.…”

“…MPIIGaze [18] datasets Accuracy = 62% Omori and Shima [14] They presented an eye contact detection method for images of the two-eye region based on the combined use of a pre-trained CNN and SVM Accuracy = 91.04%…”

Classification of Potato Leaf Diseases Using Convolutional Neural Network

“…Based on several studies, it shows that CNN has a better level of accuracy on image data. The Convolutional Neural Network method is very popular in deep learning circles because CNN extracts features from input in the form of images and then changes the dimensions of the image to be smaller without changing the characteristics of the image (Omori and Shima, 2020) Therefore, this research implemented the CNN method as a classification of potato leaf diseases. It is hoped that the CNN method will be useful for identifying potato leaf diseases so that it can reduce the number of diseases on potato leaves.…”

Literature Study of Convolutional Neural Network Algorithm for Batik Classification

“…Convolutional neural network algorithms are very popular among deep learning because the most important factor is in terms of eliminating feature extraction that can be trained according to the suitability of the task to recognize new objects that are likely to build an existing network. In addition, CNN has several other models, namely CNN with 1 conventional layer, CNN with 2 layers, CNN with 3 layers, and CNN with 4 layers (Omori & Shima, 2020).…”