Learning people co-occurrence relations by using relevance feedback for retrieving group photos

Abstract

This paper proposes an image retrieval method which retrieves images of a specific person from group photos. Many query-by-example methods have focused only on the visual features of the queried person. However, since socially related people such as family and friends are often taken photos together, their co-occurrence relations can be useful information. Thus, we propose an image retrieval method which uses the visual features of not only the queried person but also those who co-occur with the queried person in the same images. Relevance feedback is used to learn who co-occur with the queried person, their faces, and how strong their co-occurrence relations are. When retrieving the images of 19 persons in total from 158 images, after five feedback iterations, the recall rate of 50% was obtained by considering the people co-occurrence relations, as against 33% when considering only the queried person. With human errors in giving relevance feedback, the recall rate still improved to 40%.

Detection of photos from the same event captured by distinct cameras

“…Shimizu et al [13] utilizaram a co-ocorrência de pessoas para buscar um grupo de fotografias em que uma pessoa tem alta probabilidade de estar presente. Entretanto, não considera a detecção de eventos nas fotografias.…”

Improving face recognition with genealogical and contextual data

“…This is unlikely in historical family photos, with fewer photos being taken in the past. The use of identity co-occurrence within photo collections has been explored in many studies [45,15,11,52,27,35]. Shimizu et al's basic use of co-occurrence provides promising results, but requires manual confirmation of photograph relevance [45].…”

“…The use of identity co-occurrence within photo collections has been explored in many studies [45,15,11,52,27,35]. Shimizu et al's basic use of co-occurrence provides promising results, but requires manual confirmation of photograph relevance [45]. The use of context acquired solely from co-occurrence in photos within a social network, without any visual feature analysis, has provided good accuracy [15,18,47]; however these solutions are designed for use within a large scale online social network rather than in a personal collection.…”

Classification based group photo retrieval with bag of people features

“…Therefore, we propose Bag of People (BoP) features to represent both the facial appearances of persons and the co-occurrence relations among these facial appearances in the same images without identifying each person in the image. Further, we use relevance feedback, which is often used in order to bridge the semantic gap between low-level features and the semantic content of images [8,10,11,12], to adopt semi-supervised learning methods [1,13]. A user can specify images containing the target person to label them as relevant images and others as irrelevant images, and by using these labeled images as positive and negative training samples respectively, the proposed method can train a classifier to classify images into two categories, images containing the target person and other images.…”