Modelling Visual Objects Invariant to Depictive Style

Abstract

Representing visual objects is an interesting open question of relevance to many important problems in Computer Vision such as classification and location. State of the art allows thousands of visual objects to be learned and recognised, under a wide range of variations including lighting changes, occlusion, point of view, and different object instances. Only a small fraction of the literature addresses the problem of variation in depictive style (photographs, drawings, paintings etc.), yet considering photographs and artwork on equal footing is philosophically appealing and of true practical significance.This paper describes a model for visual object classes that is learnable and which is able to classify over a broad range of depictive styles. The model is a graph in which simple shapes label region nodes. We use our model to classify twenty classes in CalTech 256, each class augmented by additional images to increase the variance in style. When compared to a Bag of Words classifier and to a structure only based classifier, our results show a significant increase in robustness to variance in depictive style.

Learning Graphs to Model Visual Objects across Different Depictive Styles

“…These methods worked well for matching visually similar images, but neither are capable of modeling object categories with high diversity. The work most similar to own in motivation and method is a graph based approach proposed in [32]. They use a hierarchical graph model to obtain a coarse-to-fine arrangement of parts, whereas we use a single layer.…”

“…Experimental results clearly indicate that our mixture model outperforms state of the art methods which attempt to characterize all depiction styles in a monolithic model. We also made tests on some of the cross-domain literature we cited such as [25,32] and a method that is not depend on photometric appearance, using the edgelets [12]. A mixture-of-parts method [33] is also tested.…”

The Art of Detection

“…Our previous work [13] demonstrates that this patch-based method can be extended to object categories in paintings beyond the instance matching of [5]. Others [41,42] have considered the wider problem of generalising across many depictive styles (e.g. photo, cartoon, painting) by building a depiction-invariant graph model.…”

Cross-depiction problem: Recognition and synthesis of photographs and artwork

“…This suggests structural and spatial relations are important to cross-depiction; but the experiments are too limited to be conclusive and later tests on a larger dataset in Ref. [34] yields accuracies of around 20% (see Table 2). This suggests space and structure are important, but are insufficiently rich.…”

“…Shrivista et al [33] show that an exemplar SVM trained on a huge database is capable of classification of both photographs and artwork. A less computationally intensive approach has been proposed [34] using a hierarchical graph model to obtain a coarse-to-fine arrangement of parts with nodes labelled by qualitative shape [35]. Wu et al [36] address the cross-depiction problem using a deformable model; they use a fully connected graph with learned weights on nodes (the importance of nodes to discriminative classification), on edges (by analogy, the stiffness of a spring connecting parts), and multiple node labels (to account to different depictions); a method tested on 50 categories.…”