Iterative classification for multiple target attributes

Abstract

Many real-world applications require the simultaneous prediction of multiple target attributes. The techniques currently available for these problems either employ a global model that simultaneously predicts all target attributes or rely on the aggregation of individual models, each predicting one target. This paper introduces a novel solution. Our approach employs an iterative classification strategy to exploit the relationships among multiple target attributes to achieve higher accuracy. The computation scheme is developed as a wrapper in which many standard single-target classification algorithms can be simply "plugged-in" to simultaneously predict multiple targets. An empirical evaluation using eight data sets shows that the proposed method outperforms 1) an approach that constructs independent classifiers for each target, 2) a multitask neural network method, and 3) ensembles of multi-objective decision trees in terms of simultaneously predicting all target attributes correctly.

A Deep Learning Model for Structured Outputs with High-order Interaction

“…(3) a traditional neural network, i.e., the Multiple Layer Perceptron (MLP) with one hidden layer and multiple output nodes; (4) the so-called multivariate multiple regression (denoted as Mul-tivariateReg), which takes into account the correlations among the multiple targets using a matrix computation; (5) an approach that stacks the MultivariateReg on top of the MLP (denoted MLP-MultivariateReg); and (6) the Gaussian Conditional Random fields (GaussianCRF) [4,13,14], in which the outputs from a MLP were used as the CRF's node features, and the square of the distance between two target variables was modeled by an edge feature. In our experiments, all the parameters of these baselines have been carefully tuned.…”

“…Problems of predicting structured output span a wide range of fields, including natural language understanding, speech processing, bioinfomatics, image processing, and computer vision, amongst others. Structured learning or prediction has been approached with many different models [1,5,8,9,12], such as graphical models [7], large margin-based approaches [17], and conditional restricted Boltzmann machines [11]. Compared with structured label classification, structured output regression is a less explored topic in both the machine learning and data mining community.…”

An Efficient Feature Subset Selection Algorithm for Classification of Multidimensional Dataset

“…The solution space, which is exponential in the number of target attributes, becomes enormous, even with a limited number of target attributes. The relationships between the target attributes can add a level of complexity that needs to be taken into account [ 12 ].…”