An Optimization Framework for Combining Ensembles of Classifiers and Clusterers with Applications to Nontransductive Semisupervised Learning and Transfer Learning

Abstract

An optimization framework for combining ensembles of classifiers and clusterers with applications to nontransductive semisupervised learning and transfer learning. Data, New York, v.9, n.1, p Unsupervised models can provide supplementary soft constraints to help classify new "target" data because similar instances in the target set are more likely to share the same class label. Such models can also help detect possible differences between training and target distributions, which is useful in applications where concept drift may take place, as in transfer learning settings. This article describes a general optimization framework that takes as input class membership estimates from existing classifiers learned on previously encountered "source" (or training) data, as well as a similarity matrix from a cluster ensemble operating solely on the target (or test) data to be classified, and yields a consensus labeling of the target data. More precisely, the application settings considered are nontransductive semisupervised and transfer learning scenarios where the training data are used only to build an ensemble of classifiers and are subsequently discarded before classifying the target data. The framework admits a wide range of loss functions and classification/clustering methods. It exploits properties of Bregman divergences in conjunction with Legendre duality to yield a principled and scalable approach. A variety of experiments show that the proposed framework can yield results substantially superior to those provided by naïvely applying classifiers learned on the original task to the target data. In addition, we show that the proposed approach, even not being conceptually transductive, can provide better results compared to some popular transductive learning techniques. ACM Transactions on Knowledge Discovery from

Combining Classification and Clustering for Tweet Sentiment Analysis

“…To take advantage of both labeled and unlabeled data, several researches have designed ways of combining classifiers and clusterers [22], [37], [38], [39], [40], [41], [21], [42], [43]. Acharya et al [21] and Gao et al [43], in particular, deal with the combination of a handful of classifiers and clusterers with the ultimate goal of classifying new data.…”

“…Coletta et al [22] used a Squared Loss (SL) function in the optimization algorithm introduced in [21]. Such an algorithm, named C 3 E-SL, has presented attractive empirical results, besides being computationally efficient in practice [22], [37], [21].…”

Abordagens para combinar classificadores e agrupadores em problemas de classificação

“…O presente trabalho é focado na agregação de algoritmos de aprendizado de máquina. Em particular, o objetivo é conciliar os benefícios do uso de dados rotulados e não rotulados, a exemplo de abordagens relacionadas, as quais se baseiam na combinação de classificadores e agrupadores (GAN et al, 2013;VERMA, 2013;DUVAL-POO et al, 2012;QIAN;CAI, 2012;SOARES;YAO, 2012;RAHMAN, 2012;ACHARYA et al, 2014;ACHARYA et al, 2011;ZHANG, 2010;GAO et al, 2013;GAO et al, 2009).…”

“…Outras pesquisas têm explorado métodos que possam realizar simultaneamente supervisão e não supervisão, de maneira que tanto a classificação quanto o agrupamento possam se beneficiar do processo de aprendizado (QIAN;CAI, 2012;ZHANG, 2010;. Alguns trabalhos têm focado na agregação de diferentes classificadores e diferentes agrupadores para se obter uma classificação consolidada dos dados (ACHARYA et al, 2014;ACHARYA et al, 2011;GAO et al, 2013;GAO et al, 2009) figurando-se como abordagens bem próximas do interesse desta tese. Em particular, Acharya et al (2011) introduziram o algoritmo C 3 E (Consensus between Classification and Clustering Ensembles), que classifica novos dados a partir do processamento de dois modelos: um modelo resultante da agregação de classificadores (abordados na Seção 1.1) e outro resultante da agregação de agrupadores (discutidos na Seção 1.2).…”

Algoritmos evolutivos para modelos de mistura de gaussianas em problemas com e sem restrições

“…Algoritmos para aprendizado semissupervisionado têm sido aplicados com êxito em diversas áreas, tais como classificação de documentos (Nigam et al, 2000), detecção de pistas de estrada via GPS (Wagstaff et al, 2001), identificação de objetos em vídeos (Oyama e Tanaka, 2006) e preservação de privacidade em dados (Basu et al, 2008), e consequentemente, se constituem uma frente de pesquisa ativa atualmente (Wang et al, 2013;Li e Zemel, 2014;Acharya et al, 2014).…”