A Symmetric Loss Perspective of Reliable Machine Learning

Abstract

When minimizing the empirical risk in binary classification, it is a common practice to replace the zero-one loss with a surrogate loss to make the learning objective feasible to optimize. Examples of well-known surrogate losses for binary classification include the logistic loss, hinge loss, and sigmoid loss. It is known that the choice of a surrogate loss can highly influence the performance of the trained classifier and therefore it should be carefully chosen. Recently, surrogate losses that satisfy a certain symmetric condition (aka., symmetric losses) have demonstrated their usefulness in learning from corrupted labels. In this article, we provide an overview of symmetric losses and their applications. First, we review how a symmetric loss can yield robust classification from corrupted labels in balanced error rate (BER) minimization and area under the receiver operating characteristic curve (AUC) maximization. Then, we demonstrate how the robust AUC maximization method can benefit natural language processing in the problem where we want to learn only from relevant keywords and unlabeled documents. Finally, we conclude this article by discussing future directions, including potential applications of symmetric losses for reliable machine learning and the design of non-symmetric losses that can benefit from the symmetric condition. IntroductionModern machine learning methods such as deep learning typically require a large amount of data to achieve desirable performance [Schmidhuber, 2015;Goodfellow et al., 2016]. However, it is often the case that the labeling process is costly and time-consuming. To mitigate this problem, one may consider collecting training labels through crowdsourcing [Dawid and Skene, 1979;Kittur et al., 2008], which is a popular approach and has become more convenient in the recent years [