SVM+ regression and multi-task learning

Feng, Chao; Cherkassky, Vladimir

doi:10.1109/ijcnn.2009.5178650

Cited by 29 publications

(12 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The proposed approach can be used in the context of any regression algorithm and the experimental evaluation using regression SVMs provided satisfactory results. For more details on kernel methods and SVMs for multi-task learning, we refer the reader to [64,75,76,77] and the references therein.…”

Section: Methods For Multi-target Predictionmentioning

confidence: 99%

Tree ensembles for predicting structured outputs

Kocev

Vens

Struyf

et al. 2013

Pattern Recognition

242

261

View full text Add to dashboard Cite

In this article, we address the task of learning models for predicting structured outputs. We consider both global and local prediction of structured outputs, the former based on a single model that predicts the entire output structure and the latter based on a collection of models, each predicting a component of the output structure. We use ensemble methods and apply them in the context of predicting structured outputs. We propose to build ensemble models consisting of predictive clustering trees, which generalize classification trees: these have been used for predicting different types of structured outputs, both locally and globally. More specifically, we develop methods for learning two types of ensembles (bagging and random forests) of predictive clustering trees for global and local prediction of different types of structured outputs. The types of outputs considered correspond to different predictive modelling tasks: multi-target regression, multi-target classification, and hierarchical multi-label classification. Each of the combinations can be applied both in the context of global prediction (producing a single ensemble) or local prediction (producing a collection of ensembles). We conduct an extensive experimental evaluation across a range of benchmark datasets for each of the three types of structured outputs. We compare ensembles for global and local prediction, as well as single trees for global prediction and tree collections for local prediction, both in terms of predictive performance and in terms of efficiency (running times and model complexity). The results show that both global and local tree ensembles perform better than the single model counterparts in terms of predictive power. Global and local tree ensembles perform equally well, with global ensembles being more efficient and producing smaller models, as well as needing fewer trees in the ensemble to achieve the maximal performance.

show abstract

Section: Methods For Multi-target Predictionmentioning

confidence: 99%

Tree ensembles for predicting structured outputs

Kocev

Vens

Struyf

et al. 2013

Pattern Recognition

242

261

View full text Add to dashboard Cite

show abstract

“…In Ref , Cai and Cherkassky described a new methodology for regression problems, combining Vapnik's SVM+ regression method and the MTL setting. SVM+, also known as learning with structured data, extends the standard SVM regression by taking into account the group information available in the training data.…”

Section: Multi‐output Regressionmentioning

confidence: 99%

A survey on multi‐output regression

Borchani

Varando

Bielza

et al. 2015

WIREs Data Min & Knowl

449

319

View full text Add to dashboard Cite

In recent years, a plethora of approaches have been proposed to deal with the increasingly challenging task of multi-output regression. This paper provides a survey on state-of-the-art multi-output regression methods, that are categorized as problem transformation and algorithm adaptation methods. In addition, we present the mostly used performance evaluation measures, publicly available data sets for multi-output regression real-world problems, as well as open-source software frameworks.

show abstract

“…The LUPI paradigm has also been compared to the problem of structured or multi-task learning in both the classification [20] as well as the regression settings [5]. The multi-task learning framework considers problems where training data can naturally be separated into several groups, which can in turn be used to perform a number of individual model selections.…”

Section: Learning Using Privileged Informationmentioning

confidence: 99%

Privileged information for data clustering

Feyereisl

Aickelin

2012

Information Sciences

View full text Add to dashboard Cite

Many machine learning algorithms assume that all input samples are independently and identically distributed from some common distribution on either the input space X, in the case of unsupervised learning, or the input and output space X × Y in the case of supervised and semi-supervised learning. In the last number of years the relaxation of this assumption has been explored and the importance of incorporation of additional information within machine learning algorithms became more apparent. Traditionally such fusion of information was the domain of semi-supervised learning. More recently the inclusion of knowledge from separate hypothetical spaces has been proposed by Vapnik as part of the supervised setting. In this work we are interested in exploring Vapnik's idea of 'master-class' learning and the associated learning using 'privileged' information, however within the unsupervised setting.Adoption of the advanced supervised learning paradigm for the unsupervised setting instigates investigation into the difference between privileged and technical data. By means of our proposed aRi-MAX method stability of the K-Means algorithm is improved and identification of the best clustering solution is achieved on an artificial dataset. Subsequently an information theoretic dot product based algorithm called P-Dot is proposed. This method has the ability to utilize a wide variety of clustering techniques, individually or in combination, while fusing privileged and technical data for improved clustering. Application of the P-Dot method to the task of digit recognition confirms our findings in a real-world scenario.

show abstract

SVM+ regression and multi-task learning

Cited by 29 publications

References 6 publications

Tree ensembles for predicting structured outputs

Tree ensembles for predicting structured outputs

A survey on multi‐output regression

Privileged information for data clustering

Contact Info

Product

Resources

About