Making Visual Object Categorization More Challenging: Randomized Caltech-101 Data Set

Abstract

Visual object categorization is one of the most active research topics in computer vision, and Caltech-101 data set is one of the standard benchmarks for evaluating the method performance. Despite of its wide use, the data set has certain weaknesses: i) the objects are practically in a standard pose and scale in the middle of the images and ii) background varies too little in certain categories making it more discriminative than the foreground objects. In this work, we demonstrate how these weaknesses bias the evaluation results in an undesired manner. In addition, we reduce the bias effect by replacing the backgrounds with random landscape images from Google and by applying random Euclidean transformations to the foreground objects. We demonstrate how the proposed randomization process makes visual object categorization more challenging improving the relative results of methods which categorize objects by their visual appearance and are invariant to pose changes. The new data set is made publicly available for other researchers.

From pattern recognition methods to machine vision applications

“…It is important to observe who knows the ground truth, and why and how. To facilitate these steps in general, a lot of research methods exist: randomized hough transform (RHT) for geometric primitives detection [4][5][6][7][8][9][10][11][12][13], Gabor filtering for object detection [14][15][16][17][18], Gaussian mixture models for object classification [19,20], SOMand PCA-based image compression and representation of spectral images [21][22][23], surface analysis for 2D and 3D images [24][25][26], unsupervised methods for visual object categorization (VOC) [27][28][29][30][31], tracking methods for computer vision [32][33][34] It must be considered whether there are challenges to be expected in imaging and whether multimodal information is needed.…”

“…Different areas, that is, objects, are considered to affect the quality differently. We have generated the Randomized Caltech-101 image set [28] (with the known ground truth) and the Abstract image set [30], including human opinions as "the ground truth" [46] to test the quality of our approach. considered in Figure 11.8 where unsupervised categorization means that object categories are not known beforehand, and thus the most important ones are detected without supervision.…”

Semantic-enriched visual vocabulary construction in a weakly supervised context

“…Experiments were performed on the Caltech101 [11] and RandCaltech101 [23] datasets. Caltech101 contains 9144 images, most of them in medium resolution (300×300 pixels).…”

“…RandCaltech101 is obtained from Caltech101 by randomly modifying the backgrounds and the posture (position, orientation) of objects. It has been shown [23] that classification is more challenging on RandCaltech101 than on Caltech101.…”

Local Feature Based Unsupervised Alignment of Object Class Images

“…Moreover, some classes have virtually no background or the background remains the same. These problems make the data set bad for comparing alignment methods and for this reason, we also report the results for the recently published randomised Caltech-101 (r-Caltech-101) [15]. In r-Caltech-101 the backgrounds have been replaced with random Google landscape images and the objects transformed to random poses (scale, translation, rotation).…”