Few-Shot Personalized Saliency Prediction Based on Adaptive Image Selection Considering Object and Visual Attention

Abstract

A few-shot personalized saliency prediction based on adaptive image selection considering object and visual attention is presented in this paper. Since general methods predicting personalized saliency maps (PSMs) need a large number of training images, the establishment of a theory using a small number of training images is needed. To tackle this problem, although finding persons who have visual attention similar to that of a target person is effective, all persons have to commonly gaze at many images. Thus, it becomes difficult and unrealistic when considering their burden. On the other hand, this paper introduces a novel adaptive image selection (AIS) scheme that focuses on the relationship between human visual attention and objects in images. AIS focuses on both a diversity of objects in images and a variance of PSMs for the objects. Specifically, AIS selects images so that selected images have various kinds of objects to maintain their diversity. Moreover, AIS guarantees the high variance of PSMs for persons since it represents the regions that many persons commonly gaze at or do not gaze at. The proposed method enables selecting similar users from a small number of images by selecting images that have high diversities and variances. This is the technical contribution of this paper. Experimental results show the effectiveness of our personalized saliency prediction including the new image selection scheme.

Few-Shot Personalized Saliency Prediction using Person Similarity based on Collaborative Multi-Output Gaussian Process Regression

“…Sim↑ KLdiv↓ CC↑ Signature [1] 0.412 8.04 0.413 GBVS [2] 0.447 6.89 0.437 Itti [3] 0.391 9.04 0.322 SalGAN [4] 0.569 3.56 0.635 Baseline1 [29] 0.503 4.13 0.597 Baseline2 [30] 0.417 7.64 0.401 FPSP based on similarity [18] [23]. Among the images in the dataset, 500 images were randomly selected as test images, and the remaining 1100 images were used as training images.…”

“…In the beginning of the proposed method, the multi-task CNN is trained in order to predict the PSMs of persons whom large amount of gaze data are available. Next, we select some effective images that the target persons should gaze at based on AIS [18] and measure his/her gaze data. Note that we assume that the target person and other persons commonly gaze at these selected images.…”

“…The AIS scheme proposed in [18] for selecting the common images that the target person should gaze at is explained in this subsection. In the AIS scheme, we effectively select some images for the PSM prediction by focusing the PSM variation for objects.…”

“…By using selected images and corresponding gaze data, the similarity of gaze tendency between the target person and other persons can be calculated. Actually, the previous study based on adaptive image selection (AIS) has tried to predict the PSM of the target person by using similarities invariant to target images [18]. However, since the gaze data or PSM is related to the visual stimuli of the image [12], the similarity between persons may vary from image to image.…”

Estimation of Interest Levels From Behavior Features via Tensor Completion Including Adaptive Similar User Selection

“…Furthermore, studies using biological information are faced with the limitation of performance improvement since the number of features obtained from each user is limited. To solve the above problem, recent studies [17], [18] positively adopt not only the target user's features but also other users' features based on the intention that if users have biological information similar to the target user, their responses such as interests and flavors are also similar. However, when using other users, since users irrelevant to the target user often effect on the performance degradation, similar user selection should be considered.…”