Finding pictures of objects in large collections of images

Abstract

Retrieving images from very large collections using image content as a key is becoming an important problem. Users prefer to ask for pictures using notions of content that are strongly oriented to the presence of objects, which are quite abstractly defined. Computer programs that implement these queries automatically are desirable but are hard to build because conventional object recognition techniques from computer vision cannot recognize very general objects in very general contexts. This paper describes an approach to object recognition structured around a sequence of increasingly specialized grouping activities that assemble coherent regions of image that can be shown to satisfy increasingly stringent constraints. The constraints that are satisfied provide a form of object classification in quite general contexts. This view of recognition is distinguished by far richer involvement of early visual primitives, including color and texture; the ability to deal with rather general objects in uncontrolled configurations and contexts; and a satisfactory notion of classification. These properties are illustrated with three case studies: one demonstrates the use of descriptions that fuse color and spatial properties; one shows how trees can be described by fusing texture and geometric properties; and one shows how this view of recognition yields a program that can tell, quite accurately, whether a picture contains naked people or not.

Learning Spatial Context: Using Stuff to Find Things

“…These two lines of work have made significant progress on the problems of identifying "things" and "stuff," respectively. The important differentiation between these two classes of visual objects is summarized in Forsyth et al [4] as: Example detections from the satellite dataset that demonstrate context. Classifying using local appearance only, we might think that both windows at left are cars.…”

Massive City-Scale Surface Condition Analysis Using Ground and Aerial Imagery

“…The street images are used for appearance-based recognition of 'stuff' [41] described in Section 3.1. The results of pixel-wise regression are then projected onto the ground plane as an input feature for our city-scale GP regressor.…”

Protecting Privacy from Continuous High-Resolution Satellite Surveillance

“…They include (1) filtering of images of naked people using a skin filter and a human figure grouper [3,4], and (2) using a content-based feature vector indexing where an image is matched against a small number of feature vectors obtained from a training database [16,17]. However, these approaches filter all images that match a set of criteria, but do not provide controlled access that facilitates access to images for legitimate users.…”