Active zero-shot learning: a novel approach to extreme multi-labeled classification

Abstract

Big data bring a huge volume of data in a great speed and in many formats with extremely many labels and concepts to be modeled and predicted, such as in text and image tagging, online advertisement placement, recommendation systems, NLP. This emerging issue of big data is termed "extreme multi-labeled classification" (XMLC) and is challenging due to the time, space and sample complexity in predictive model training and testing. We first define general XMLC and then categorize and review recent methods based on two specific forms of XMLC. We propose a novel method called active zero-shot learning to reduce the above complexities. Since the performance of the unseen class prediction largely depends on the seen classes that have labeled data, we challenge the critical and yet often overlooked assumption that the labeled data can only be passively acquired. We propose a new learning paradigm aiming at accurate predictions of a large number of unseen labels using labeled data from only an intelligently selected small set of seed classes with the help of external knowledge. We further demonstrate that the proposed strategy has desirable probabilistic properties to facilitate unseen classes prediction. Experiments on 4 datasets demonstrate that the proposed algorithm is superior to a wide spectrum of baselines. Based on our findings, we point out several critical and promising future directions in XMLC.

What Can Knowledge Bring to Machine Learning? -- A Survey of Low-shot Learning for Structured Data

“…The query is the basic operation to access structured data. For LSL query modeling on structured data, the basic strategy consists of actively selecting query and labeled data to create seen classes so that unseen classes can be predicted [238,239]. [99] extend the training ability of meta-learning framework by effectively creating pseudo-tasks with the help of a relevance function, then solve the semantic parsing problem that maps natural language questions to SQL statements.…”

SEVEN: Deep Semi-supervised Verification Networks

“…Another line of work that handles a large number of categories is extreme multi-label learning [Xie and Philip, 2017]. The most popular assumption is that all classes have sufficient amount of labeled data, and this is clearly different from our problem setting.…”

“…The goal of metric learning is to learn a distance metric such that samples in any negative pair are far away and those in any positive pair are close. Many of the existing approaches [Sun et al, 2017;Oh Song et al, 2016;Zagoruyko and Komodakis, 2015;Hu et al, 2014] learn a linear or nonlinear transformation that maps the data to a new space where the distance metric satisfies the above requirements. However, these methods do not address the large number of categories with scarce supervision information.…”

Generalized Zero-Shot Extreme Multi-label Learning

“…ZestXML outperforms all these approaches by up to 14% in prediction accuracy due to better modeling and access to more label choices from unseen ones. Zero-shot multi-label learning: Although zero-shot classification is a highly researched area, most of the existing work concerns multi-class learning [15,25,27,29,33,38,43,51,59] with only a handful of multi-label learning approaches [7,14,17,18,30,41,50,69]. These existing ZML algorithms are designed for 100s-1000s of labels and don't scale to 1M labels.…”