Privileged information for data clustering

Abstract

Many machine learning algorithms assume that all input samples are independently and identically distributed from some common distribution on either the input space X, in the case of unsupervised learning, or the input and output space X × Y in the case of supervised and semi-supervised learning. In the last number of years the relaxation of this assumption has been explored and the importance of incorporation of additional information within machine learning algorithms became more apparent. Traditionally such fusion of information was the domain of semi-supervised learning. More recently the inclusion of knowledge from separate hypothetical spaces has been proposed by Vapnik as part of the supervised setting. In this work we are interested in exploring Vapnik's idea of 'master-class' learning and the associated learning using 'privileged' information, however within the unsupervised setting.Adoption of the advanced supervised learning paradigm for the unsupervised setting instigates investigation into the difference between privileged and technical data. By means of our proposed aRi-MAX method stability of the K-Means algorithm is improved and identification of the best clustering solution is achieved on an artificial dataset. Subsequently an information theoretic dot product based algorithm called P-Dot is proposed. This method has the ability to utilize a wide variety of clustering techniques, individually or in combination, while fusing privileged and technical data for improved clustering. Application of the P-Dot method to the task of digit recognition confirms our findings in a real-world scenario.

Multimodal Emotion Recognition Using the Symmetric S-ELM-LUPI Paradigm

“…Learning Using Privileged Information (LUPI) paradigm has been largely applied with Support Vector Machine plus (SVM+) algorithm. Feyereisl et al [7] has worked on the importance and incorporation of privileged information in cluster analysis and their method has improved the clustering performance. Ji et al [8] have proposed a multi-task multi-class by learning using privileged information on support vector machines.…”

One Method to Reduce Data Classification Using Weighting Technique in SVM +

“…In other words, any solution of SVM + is a WSVM solution by selecting the appropriate weight. But it is essential to note that any non-trivial solution of SVM + is unique, in contrast to other solutions that may be non-unique so WSVM and SVM + are not equivalent (Carlos et al, 2014;Maksim et al, 2014;Feyereisl & Aickelin, 2012).…”

Aprendizado de máquina com informação privilegiada: abordagens para agrupamento hierárquico de textos

“…O paradigma LUPI foi proposto originalmente para estender o algoritmo de aprendizado supervisionado SV M (Support Vector Machine) (Cortes e Vapnik, 1995) para o SV M + (Vapnik e Vashist, 2009), que incorpora informação privilegiada visando melhorar o desempenho de modelos preditivos usados em problemas de classificação e regressão. Apesar da relevância do tema, ainda há poucos estudos na literatura que exploram o paradigma LUPI em tarefas de agrupamento (Feyereisl e Aickelin, 2012).…”

“…Em muitos problemas de aprendizado de máquina há informação adicional a respeito dos dados que pode ser utilizada durante o treinamento do modelo inicial. O paradigma LUPI permite melhorar a tarefa de aprendizado considerando esta informação privilegiada durante a indução dos modelos, tanto de classificação (Vapnik e Vashist, 2009) quanto de agrupamento (Feyereisl e Aickelin, 2012). Estudos recentes indicam que a incorporação da informação privilegiada permite a obtenção de modelos mais robustos, no qual o modelo gerado (pela combinação de informação técnica e privilegida) deve classificar um grande conjunto de exemplos descritos somente pela informação técnica (Pechyony e Vapnik, 2010.…”

“…A abordagem aRi-MAX utiliza os conceitos de seleção de modelos para agrupamento de dados com uso de informação privilegiada (Feyereisl e Aickelin, 2012). A ideia geral do aRi-MAX é selecionar, dentro um conjunto de modelos, o modelo que obtém maior concordância nos dois espaços de características χ (informação técnica) e χ * (informação privilegiada).…”