SUN attribute database: Discovering, annotating, and recognizing scene attributes

Patterson, Geneviève; Hays, James

doi:10.1109/cvpr.2012.6247998

Cited by 712 publications

(536 citation statements)

References 19 publications

Supporting

Mentioning

530

Contrasting

Unclassified

Order By: Relevance

“…4). The work in these sections largely appears in a previous publication (Patterson and Hays 2012). Section 5 contains significantly expanded work, and Sects.…”

Section: Paper Outlinementioning

confidence: 98%

The SUN Attribute Database: Beyond Categories for Deeper Scene Understanding

et al. 2014

View full text Add to dashboard Cite

In this paper we present the first large-scale scene attribute database. First, we perform crowdsourced human studies to find a taxonomy of 102 discriminative attributes. We discover attributes related to materials, surface properties, lighting, affordances, and spatial layout. Next, we build the "SUN attribute database" on top of the diverse SUN categorical database. We use crowdsourcing to annotate attributes for 14,340 images from 707 scene categories. We perform numerous experiments to study the interplay between scene attributes and scene categories. We train and evaluate attribute classifiers and then study the feasibility of attributes as an intermediate scene representation for scene classification, zero shot learning, automatic image captioning, semantic image search, and parsing natural images. We show that when used as features for these tasks, low dimensional scene attributes can compete with or improve on the state of the art performance. The experiments suggest that scene attributes are an effective low-dimensional feature for capturing high-level context and semantics in scenes.

show abstract

“…4). The work in these sections largely appears in a previous publication (Patterson and Hays 2012). Section 5 contains significantly expanded work, and Sects.…”

Section: Paper Outlinementioning

confidence: 98%

The SUN Attribute Database: Beyond Categories for Deeper Scene Understanding

et al. 2014

View full text Add to dashboard Cite

show abstract

“…The problem of detecting image state has received some prior attention. For example, researchers have worked on recognizing image "attributes" (e.g., [10], [24], [23], [11]), which sometimes include object and scene states. However, most of this work has dealt with one image at a time and has not extensively catalogued the state variations that occur in an entire image class.…”

Section: Introductionmentioning

confidence: 99%

Discovering states and transformations in image collections

Isola

Lim

Adelson

2015

2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

181

161

View full text Add to dashboard Cite

Objects in visual scenes come in a rich variety of transformed states. A few classes of transformation have been heavily studied in computer vision: mostly simple, parametric changes in color and geometry. However, transformations in the physical world occur in many more flavors, and they come with semantic meaning: e.g., bending, folding, aging, etc. The transformations an object can undergo tell us about its physical and functional properties. In this paper, we introduce a dataset of objects, scenes, and materials, each of which is found in a variety of transformed states. Given a novel collection of images, we show how to explain the collection in terms of the states and transformations it depicts. Our system works by generalizing across object classes: states and transformations learned on one set of objects are used to interpret the image collection for an entirely new object class.

show abstract

“…(a) Scenes: We use the SUN Attributes dataset [37] that consists of indoor and outdoor scenes along with pre-trained attribute classifiers for 102 attributes like natural, open, enclosed, warm, etc. Scene categories in the SUN dataset [1] are organized according to a hierarchy, where the first level has super-ordinate categories like indoor and outdoor scenes, the second level has basic-level categories like sports, transportation, desert, etc., and the third (last) level has more than 700 fine-grained categories.…”

Section: Methodsmentioning

confidence: 99%

Interactively Guiding Semi-Supervised Clustering via Attribute-Based Explanations

Lad

Parikh

2014

Computer Vision – ECCV 2014

View full text Add to dashboard Cite

(ABSTRACT)Unsupervised image clustering is a challenging and often ill-posed problem. Existing image descriptors fail to capture the clustering criterion well, and more importantly, the criterion itself may depend on (unknown) user preferences. Semi-supervised approaches such as distance metric learning and constrained clustering thus leverage user-provided annotations indicating which pairs of images belong to the same cluster (must-link) and which ones do not (cannot-link). These approaches require many such constraints before achieving good clustering performance because each constraint only provides weak cues about the desired clustering.In this work, we propose to use image attributes as a modality for the user to provide more informative cues. In particular, the clustering algorithm iteratively and actively queries a user with an image pair. Instead of the user simply providing a must-link/cannot-link constraint for the pair, the user also provides an attribute-based reasoning e.g. "these two images are similar because both are natural and have still water" or "these two people are dissimilar because one is way older than the other". Under the guidance of this explanation, and equipped with attribute predictors, many additional constraints are automatically generated.We demonstrate the effectiveness of our approach by incorporating the proposed attributebased explanations in three standard semi-supervised clustering algorithms: Constrained K-Means, MPCK-Means, and Spectral Clustering, on three domains: scenes, shoes, and faces, using both binary and relative attributes.

show abstract

SUN attribute database: Discovering, annotating, and recognizing scene attributes

Cited by 712 publications

References 19 publications

The SUN Attribute Database: Beyond Categories for Deeper Scene Understanding

The SUN Attribute Database: Beyond Categories for Deeper Scene Understanding

Discovering states and transformations in image collections

Interactively Guiding Semi-Supervised Clustering via Attribute-Based Explanations

Contact Info

Product

Resources

About