Predicting human gaze using quaternion DCT image signature saliency and face detection

Abstract

We combine and extend the previous work on DCT-based image signatures and face detection to determine the visual saliency. To this end, we transfer the scalar definition of image signatures to quaternion images and thus introduce a novel saliency method using quaternion type-II DCT image signatures. Furthermore, we use MCT-based face detection to model the important influence of faces on the visual saliency using rotated elliptical Gaussian weight functions and evaluate several integration schemes. In order to demonstrate the performance of the proposed methods, we evaluate our approach on the Bruce-Tsotsos (Toronto) [2] and Cerf (FIFA) [3] benchmark eye-tracking data sets. Additionally, we present evaluation results on the BruceTsotsos data set of the most important spectral saliency approaches. We achieve state-of-the-art results in terms of the well-established area under curve (AUC) measure on the Bruce-Tsotsos data set and come close to the ideal AUC on the Cerf data set -with less than one millisecond to calculate the bottom-up QDCT saliency map.

Quaternion-Based Spectral Saliency Detection for Eye Fixation Prediction

“…Also in 2012 [3], Schauerte et al used the denition of a quaternion DCT and quaternion signatures to calculate the visual saliency and was able to outperform the real-valued approach by Hou et al [2]. This way they improved the state-ofthe-art in predicting where humans look in the presence and absence of faces.…”

“…Quaternion Discrete Cosine Transform: Similarly, it is possible to dene a quaternion discrete cosine transform: 2 )u (see, e.g., [3]). However, as can be seen when comparing Eq.…”

“…An interesting development in this area is the use of quaternions as a holistic representation to process color images as a whole [3,4]. The quaternion algebra makes it possible to process color images as a whole without the need to process the image channels separately and, in consequence, tear apart the color information 1 .…”

“…However, the evaluated spectral saliency algorithms achieve the performance at a fraction of the computational requirements of Judd's algorithm. The proposed use of the eigenaxis and eigenangle substantially improves the performance of the quaternion Fourier approach (PQFT; see [4]) and makes it the second best choice after the quaternion DCT signature saliency (QDCT; see [3]). The use of multiple scales signicantly improves the results and the use of appropriately weighted color spaces is essential to achieve outstanding performance using quaternion-based spectral saliency detection.…”

“…We systematically integrate and evaluate quaternion DCT-and FFT-based spectral saliency detection [3,4], weighted quaternion color space components [5], and the use of multiple resolutions [1]. Furthermore, we propose the use of the eigenaxes and eigenangles for spectral saliency models that are based on the quaternion Fourier transform.…”