ConceptLearner: Discovering visual concepts from weakly labeled image collections

Zhou, Bolei; Jagadeesh, Vignesh; Piramuthu, Robinson

doi:10.1109/cvpr.2015.7298756

Cited by 37 publications

(31 citation statements)

References 28 publications

(45 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In Guillaumin et al [2009] and Verbeek et al [2010], logistic regression models are built per tag to promote rare tags. In a similar spirit to , Zhou et al [2015] learn an ensemble of SVMs by treating tagged images as positive training examples and untagged images as candidate negative training examples. Using the ensemble to classify image regions generated by automated image segmentation, the authors assign tags at the image level and the region level simultaneously.…”

Section: Modelmentioning

confidence: 99%

Socializing the Semantic Gap

et al. 2016

View full text Add to dashboard Cite

Where previous reviews on content-based image retrieval emphasize what can be seen in an image to bridge the semantic gap, this survey considers what people tag about an image. A comprehensive treatise of three closely linked problems (i.e., image tag assignment, refinement, and tag-based image retrieval) is presented. While existing works vary in terms of their targeted tasks and methodology, they rely on the key functionality of tag relevance, that is, estimating the relevance of a specific tag with respect to the visual content of a given image and its social context. By analyzing what information a specific method exploits to construct its tag relevance function and how such information is exploited, this article introduces a two-dimensional taxonomy to structure the growing literature, understand the ingredients of the main works, clarify their connections and difference, and recognize their merits and limitations. For a head-to-head comparison with the state of the art, a new experimental protocol is presented, with training sets containing 10,000, 100,000, and 1 million images, and an evaluation on three test sets, contributed by various research groups. Eleven representative works are implemented and evaluated. Putting all this together, the survey aims to provide an overview of the past and foster progress for the near future.

show abstract

Section: Modelmentioning

confidence: 99%

Socializing the Semantic Gap

et al. 2016

View full text Add to dashboard Cite

show abstract

“…They show that attributes depend on features in all layers of the CNN, which will be particularly relevant to our investigation of perceptual material attributes in CNNs. ConceptLearner, proposed by Zhou et al [25] uses weak supervision, in the form of images with associated text content, to discover semantic attributes. These attributes correspond to terms within the text that appear in the images.…”

Section: Materials Perception and Convolutional Neural Networkmentioning

confidence: 99%

“…Similarly for conventional object and scene recognition, attributes like "sunset" or "natural," have also been extracted for use as independent features. Shankar et al [23] generate pseudo-labels to improve the attribute prediction accuracy of a Convolutional Neural Network, and Zhou et al [25] discover concepts from weakly-supervised image data. In both cases, the attributes are considered on their own, not within the context of higher-level categories.…”

Section: Perceptual Materials Attributes In Con-volutional Neural Netwmentioning

confidence: 99%

Recognizing Material Properties from Images

Schwartz

Nishino

2020

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Humans implicitly rely on properties of the materials that make up ordinary objects to guide our interactions. Grasping smooth materials, for example, requires more care than rough ones, and softness is an ideal property for fabric used in bedding. Even when these properties are not purely visual (softness is a physical property of the material), we may still infer the softness of a fabric by looking at it. We refer to these visually-recognizable material properties as visual material attributes. Recognizing visual material attributes in images can contribute valuable information for general scene understanding and for recognition of materials themselves. Unlike wellknown object and scene attributes, visual material attributes are local properties. "Fuzziness", for example, does not have a particular shape. We show that given a set of images annotated with known material attributes, we may accurately recognize the attributes from purely local information (small image patches). Obtaining such annotations in a consistent fashion at scale, however, is challenging. We introduce a method that allows us to solve this problem by probing the human visual perception of materials to automatically discover unnamed attributes that serve the same purpose. By asking simple yes/no questions comparing pairs of image patches, we obtain sufficient weak supervision to build a set of attributes (and associated classifiers) that, while being unnamed, serve the same function as the named attributes, such as "fuzzy" or "rough", with which we describe materials. Doing so allows us to recognize visual material attributes without resorting to exhaustive manual annotation of a fixed set of named attributes. Furthermore, we show that our automatic attribute discovery method may be integrated in the end-to-end learning of a material classification CNN framework to simultaneously recognize materials and discover their visual material attributes. Our experimental results show that visual material attributes, whether named or automatically discovered, provide a useful intermediate representation for known material categories themselves as well as a basis for transfer learning when recognizing previously-unseen categories.Index Terms-visual material attributes, human material perception, material recognition ! The authors are with the

show abstract

“…LEVAN [8] explores the sub-categories of a given concept by mining bigrams from large text corpus and using the bigrams to retrieve training images from image search engines. Recently, Zhou et al [44] use noisily tagged Flickr images to train concept detectors, but do not consider the semantic similarity among different tags. Our VCD framework is able to generate the concept vocabulary for them to learn detectors.…”

Section: Related Workmentioning

confidence: 99%

Automatic Concept Discovery from Parallel Text and Visual Corpora

Sun

Gan

Nevatia

2015

2015 IEEE International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Humans connect language and vision to perceive the world. How to build a similar connection for computers? One possible way is via visual concepts, which are text terms that relate to visually discriminative entities. We propose an automatic visual concept discovery algorithm using parallel text and visual corpora; it filters text terms based on the visual discriminative power of the associated images, and groups them into concepts using visual and semantic similarities. We illustrate the applications of the discovered concepts using bidirectional image and sentence retrieval task and image tagging task, and show that the discovered concepts not only outperform several large sets of manually selected concepts significantly, but also achieves the stateof-the-art performance in the retrieval task.

show abstract

ConceptLearner: Discovering visual concepts from weakly labeled image collections

Cited by 37 publications

References 28 publications

Socializing the Semantic Gap

Socializing the Semantic Gap

Recognizing Material Properties from Images

Automatic Concept Discovery from Parallel Text and Visual Corpora

Contact Info

Product

Resources

About