Invariant operators, small samples, and the bias-variance dilemma

Shi, Xiaojin; Manduchi, Roberto

doi:10.1109/cvpr.2004.1315209

Cited by 4 publications

(2 citation statements)

References 13 publications

(19 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The trick is to find a change of basis that makes our computation simple. Note that an invertible transformation in the feature space does not change the Bayes rate [30], and therefore we can compute P B (E) in the transformed space.…”

Section: Appendix Amentioning

confidence: 99%

“…With some abuse of language, we will say that a classifier is Bayesian if it is generated by an algorithm that is asymptotically unbiased at each x. much better (in terms of misclassification rate) than texture, while the opposite is true for the data set ''Trees''. We used non-normalized (r, g, b) color features in our experiments (see [30] for a comparison of normalized and non-normalized colors in terms of misclassification rate). For texture, we used standard Gabor features [23].…”

Section: Visual Featuresmentioning

confidence: 99%

See 1 more Smart Citation

On the Bayes fusion of visual features

Shi

Manduchi

2007

Image and Vision Computing

View full text Add to dashboard Cite

We consider the problem of image classification when more than one visual feature is available. In such cases, Bayes fusion offers an attractive solution by combining the results of different classifiers (one classifier per feature). This is the general form of the so-called ''naive Bayes'' approach. This paper compares the performance of Bayes fusion with respect to Bayesian classification, which is based the joint feature distribution. It is well-known that the latter has lower bias than the former, unless the features are conditionally independent, in which case the two coincide. However, as originally noted by Friedman, the low variance associated with naive Bayes estimation may mitigate the effect of its inherent bias. Indeed, in the case of small training samples, naive Bayes may outperform Bayes classification in terms of error rate.The contribution of this paper is threefold. First, we present a detailed analysis of the error rate of Bayes fusion assuming that the statistical description of the data is known. Second, we provide a qualitative justification of the small sample effect on the classifier's performance based on the bias/variance theory. Third, we present experimental results on three image data sets using color and texture features. Our experiments highlight the relationship between the error rate of the Bayes and the Bayes fusion classifiers as a function of the training sample size.

show abstract

Section: Appendix Amentioning

confidence: 99%

Section: Visual Featuresmentioning

confidence: 99%