Eye Tracking, Saliency Modeling and Human Feedback Descriptor Driven Robust Region-of-Interest Determination Technique

Abstract

The Region of interest (ROI) analysis is widely used in image analytics, video coding, computer graphics, computer vision, medical imaging, nuclear medicine, computer tomography and many more areas in medical applications. This ROI determination process using subjective method (e.g. using human vision) often differ from the objective ones (e.g. using mathematical modelling). However, there is no existing method in the literature that could provide a single decision when both method's ROI data are available. To address this limitation, in this paper, a robust algorithm is developed through a combining process of human eye tracking (subjective) and graph-based saliency modelling (objective) information to determine a more realistic ROI for a scene. To carry out this process, in one hand, a number of different independent human visual saliency factors such as pupil size, pupil dilation, central tendency, fixation pattern, and gaze plot for a group of twenty-two participants are collected by applying on a set of publicly available eighteen video sequences. On the other hand, the features of Graph based visual saliency (GBVS) highlights conspicuity in the scene. Gleaned from this two information, the proposed algorithm determines the final ROI based on some heuristics. Experimental results show that for a wide range of video sequences and compared to the existing deep learning based (MxSalNet) and depth pixel (DP) based ROI, the proposed ROI is consistent to the benchmark ROI, which was previously decided by a group video coding expert. As the subjective and objective options frequently create an ambiguity to reach a single decision on ROI, the proposed algorithm could determine an ultimate decision, which is eventually validated by experts' opinion.