Fragment-Based Learning of Visual Object Categories

Abstract

When we perceive a visual object, we implicitly or explicitly associate it with a category we know. It is known that the visual system can use local, informative image fragments of a given object, rather than the whole object, to classify it into a familiar category. How we acquire informative fragments has remained unclear. Here, we show that human observers acquire informative fragments during the initial learning of categories. We created new, but naturalistic, classes of visual objects by using a novel "virtual phylogenesis" (VP) algorithm that simulates key aspects of how biological categories evolve. Subjects were trained to distinguish two of these classes by using whole exemplar objects, not fragments. We hypothesized that if the visual system learns informative object fragments during category learning, then subjects must be able to perform the newly learned categorization by using only the fragments as opposed to whole objects. We found that subjects were able to successfully perform the classification task by using each of the informative fragments by itself, but not by using any of the comparable, but uninformative, fragments. Our results not only reveal that novel categories can be learned by discovering informative fragments but also introduce and illustrate the use of VP as a versatile tool for category-learning research.

Creating Objects and Object Categories for Studying Perception and Perceptual Learning

“…An important task in visual processing is inferring the category to which a given observed object belongs. Although the exact mechanism of this inference is unknown, there is both computational and physiological evidence 9,12,13,[30][31][32] that it involves using the information about known features of the object in the given image to infer the category of the object. Here, we will illustrate how this inferential process may work in a Bayesian framework, and how digital embryos may be useful for research in this area.…”

“…Briefly, VM, especially the digital embryo methodology, is useful because it provides a principled and flexible method for creating novel, but naturalistic 3-D objects 14 . Similarly, VP provides a principled method of creating naturalistic categories 9,10,12,13 . It is worth noting that object categories generated by VP share many features with object categories in nature, including the fact that the categories tend to be hierarchical in nature, and the feature variations within and across categories arise independently of the experimenter and the algorithms for classifying them 39 .…”

“…Objects and object categories created by these simulations can be further manipulated by various morphing methods to generate systematic variations of shape characteristics 15,16 . The VP and morphing methods can also be applied, in principle, to novel virtual objects other than digital embryos, or to virtual versions of realworld objects 9,13 . Virtual objects created in this fashion can be rendered as visual images using a conventional graphical toolkit, with desired manipulations of surface texture, illumination, size, viewpoint and background.…”

“…7,8). However, generally speaking, the existing methods have three broad types of shortcomings.First, shape variations are generally imposed by the experimenter 5,9,10 , and may therefore be different from the variability in natural categories, and optimized for a particular recognition algorithm. It would be desirable to have the variations arise independently of the externally imposed constraints.…”

“…Virtual morphogenesis (VM) creates novel, naturalistic virtual 3-D objects called 'digital embryos' by simulating the biological process of embryogenesis 14 . Virtual phylogenesis (VP) creates novel, naturalistic object categories by simulating the evolutionary process of natural selection 9,12,13 . Objects and object categories created by these simulations can be further manipulated by various morphing methods to generate systematic variations of shape characteristics 15,16 .…”