Learning from ambiguously labeled images

Abstract

In many image and video collections, we have access only to partially labeled data. For example, personal photo collections often contain several faces per image and a caption that only specifies who is in the picture, but not which name matches which face. Similarly, movie screenplays can tell us who is in the scene, but not when and where they are on the screen. We formulate the learning problem in this setting as partially-supervised multiclass classification where each instance is labeled ambiguously with more than one label. We show theoretically that effective learning is possible under reasonable assumptions even when all the data is weakly labeled. Motivated by the analysis, we propose a general convex learning formulation based on minimization of a surrogate loss appropriate for the ambiguous label setting. We apply our framework to identifying faces culled from web news sources and to naming characters in TV series and movies. We experiment on a very large dataset consisting of 100 hours of video, and in particular achieve 6% error for character naming on 16 episodes of LOST.

Large-Scale Object Classification Using Label Relation Graphs

“…To our knowledge we are the first to jointly model hierarchical and exclusive relations. By treating unobserved labels as latent variables, our approach also connects to prior work on learning from partial or incomplete labels [19,7,5].…”

Selecting training sets for support vector machines: a review

“…A generic solution for learning SVMs from weak labels was introduced by Li et al (2013)-labels are subject to the optimization. This is effective, if vectors belonging to the opposite classes form well-separated clusters in the kernel space, but this assumption may not hold in many scenarios (Cour et al 2009;Tapaswi et al 2015).…”

Cross-Lingual Part-of-Speech Tagging through Ambiguous Learning

“…This reduction of sequence labeling to multiclass classification allows us to learn a sequence model in an ambiguous setting by building on the theoretical results of Bordes et al (2010) and Cour et al (2011). The decision about the correctness of a prediction and the weight updates can be adapted to the amount of supervision information that is available.…”

“…By recasting this setting in the framework of ambiguous learning (Bordes et al, 2010;Cour et al, 2011) (Section 3), we propose an alternative learning methodology and show that it improves the state of the art performance on a large array of languages (Section 4). Our analysis of the remaining errors suggests that in cross-lingual settings, improvements of error rates can have multiple causes and should be looked at with great care (Section 4.2).…”