Labelling Image Regions Using Wavelet Features and Spatial Prototypes

Abstract

Abstract. In this paper we present an approach for image region classification that combines low-level processing with high-level scene understanding. For the low-level training, predefined image concepts are statistically modelled using wavelet features extracted directly from image pixels. For classification, features of a given test region compared with these statistical models provide probabilistic evaluations for all possible image concepts. Maximising these values themselves already leads to a classification result (label). However, in our paper they are used as an input for the high-level approach exploiting explicitly represented spatial arrangements of labels, so called spatial prototypes. We formalise the problem using Fuzzy Constraint Satisfaction Problems and Linear Programming. They provide a model with explicit knowledge that is suitable to aid the task of region labelling. Experiments performed for nearly 6000 test image regions show that combining low-level and high-level image analysis increases the labelling accuracy significantly.

An energy-based model for region-labeling

“…Both methods use spatial constraint templates obtained from labeled images. Results with FCSP and BIP in the SCEF collection show that their methods can improve the initial labeling provided by the classifier [12,13]. The improvements are more significative for the BIP formulation, which is also much more efficient than the FSCS approach.…”

“…Carsten et al proposed a fuzzy constraint satisfaction (FCSP) approach [13] and a binary integer programming (BIP) formulation [12] for improving the initial labeling provided by a classifier based on visual features. Both methods use spatial constraint templates obtained from labeled images.…”

“…Whereas satisfactory performance has been reported with this sort of methods, some single-machine techniques are highly complex (because a single model is used for C labels) while others obtain limited classification performance, thus other formulations are preferred. OVA approaches have been adopted for RL as well [10][11][12][13][14], where C binary classifiers are build (one per label) and the outputs of binary classifiers are combined to obtain a multiclass predictions. Acceptable performance has been reported with those techniques, besides OVA is more efficient than AVA and simpler than single-machine techniques.…”

“…The images are taken from the SAIAPR TC-12 benchmark [4]. of multiclass classifiers [9][10][11][12][13][14][15]. The underlying idea of these methods is to perform an initial labeling by multiclass classifiers and then apply different strategies to improve the labeling provided by the preceding classification.…”

“…The pairs of feature vectors and labels form the training set for a usual multiclass classification task, with as many classes as labels in the vocabulary. This scheme has been adopted, for example, in [9][10][11][12][13][14][15][16][17][18][19].…”