We consider the problem of learning linear classifiers when both features and labels are binary. In addition, the features are noisy, i.e., they could be flipped with an unknown probability. In Sy-De attribute noise model, where all features could be noisy together with same probability, we show that 0-1 loss (l0−1) need not be robust but a popular surrogate, squared loss (lsq) is. In Asy-In attribute noise model, we prove that l0−1 is robust for any distribution over 2 dimensional feature space. However, due to computational intractability of l0−1, we resort to lsq and observe that it need not be Asy-In noise robust. Our empirical results support Sy-De robustness of squared loss for low to moderate noise rates.
In binary classification framework, we are interested in making cost sensitive label predictions in the presence of uniform/symmetric label noise. We first observe that 0-1 Bayes classifiers are not (uniform) noise robust in cost sensitive setting. To circumvent this impossibility result, we present two schemes; unlike the existing methods, our schemes do not require noise rate. The first one uses α-weighted γ-uneven margin squared loss function, lα,usq, which can handle cost sensitivity arising due to domain requirement (using user given α) or class imbalance (by tuning γ) or both. However, we observe that lα,usq Bayes classifiers are also not cost sensitive and noise robust. We show that regularized ERM of this loss function over the class of linear classifiers yields a cost sensitive uniform noise robust classifier as a solution of a system of linear equations. We also provide a performance bound for this classifier. The second scheme that we propose is a re-sampling based scheme that exploits the special structure of the uniform noise models and uses in-class probability η estimates. Our computational experiments on some UCI datasets with class imbalance show that classifiers of our two schemes are on par with the existing methods and in fact better in some cases w.r.t. Accuracy and Arithmetic Mean, without using/tuning noise rate. We also consider other cost sensitive performance measures viz., F measure and Weighted Cost for evaluation. If the noise is adversarial then, one can interpret not requiring the noise rates as a dominant strategy and cross-validation over noise rates as a search for a dominant strategy. Also, the required restriction of the hypothesis class to the linear class can be interpreted as an implicit form of regularization. As our re-sampling scheme requires estimates of η, we provide a detailed comparative study of various η estimation methods on synthetic datasets, w.r.t. half a dozen evaluation criterion. Also, we provide understanding on the interpretation of cost parameters α and γ using different synthetic data experiments.
W.B. Yeats is a poet of great artistic honesty and integrity. In his critical statements he has candidly stated his artistic intentions and preoccupations. For such students of Yeats as have been puzzled by the artistic intentions of the poet in regard to the subject matter of his poetry, he has clarified his stance in his cryptic remark, “I remake myself” in my poetic compositions. Yeats, as every scholar knows it, is paradoxically a very complex and a simple poet. His complexity lies in his reader’s bewilderment at the contradictory artistic issues and his simplicity lies in its being a key to the unlocking of some of the ambiguities lurking in some of his poems. In his poetry, sometime he raises an issue which develops logically but the conclusion seems to stultify his explicit poetic statement, because the conclusion of the poem is deliberately made meaningfully ambiguous. His “Sailing to Byzantium” is a poem of this type of complex simplicity.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.