Perceptron and SVM learning with generalized cost models

Geibel, Peter; Brefeld, Ulf; Wysotzki, Fritz

doi:10.3233/ida-2004-8502

Cited by 13 publications

(17 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We also performed experiments with the support vector machine. In contrast to the experiments with an extended Matlab implementation of the 2-norm SVM described in [14], we used the SVMLight [32] here. This particular implementation of a 1-norm SVM allows the use of individual costs as example weights given as real values.…”

Section: Comparison With Other Algorithms Of Classification Learningmentioning

confidence: 99%

“…We also took a cost-proportional resampling method into consideration, as described in [13,14]. The resampling method we developed consists of building a new, cost-free dataset based on the original dataset with costs (see also [33,34]).…”

Section: Comparison With Other Algorithms Of Classification Learningmentioning

confidence: 99%

“…The datasets NSEP OC and MULTI OC have mainly been designed for demonstrating the qualitative behaviour of the costs-sensitive learning methods. However, for the costs-sensitive versions of DIPOL and the SVM, diagrams corresponding to the ones for CAL5 OC (Figures 4 and 5) can be found in [13,14]. The specific shape of the class borders differs: CAL5 OC computes axis-parallel borders and DIPOL OC finds separate hyperplanes in a general position, while the shape of the borders computed by the SVM is determined by the kernel used.…”

Section: Algorithmmentioning

confidence: 99%

“…One approach for using example-dependent costs has already been discussed and applied in the context of rough classifiers [11], the perceptron, piecewise linear classifiers, support vector machines [12][13][14], in the examination of concept drift [15], and in reinforcement learning and process control [16,17]. In this paper it is applied to the decision tree learning (see also [18,19]).…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A New Information Measure Based on Example-Dependent Misclassification Costs and Its Application in Decision Tree Learning

Wysotzki

Geibel

2009

Advances in Artificial Intelligence

Self Cite

View full text Add to dashboard Cite

This article describes how the costs of misclassification given with the individual training objects for classification learning can be used in the construction of decision trees for minimal cost instead of minimal error class decisions. This is demonstrated by defining modified, cost-dependent probabilities, a new, cost-dependent information measure, and using a cost-sensitive extension of the CAL5 algorithm for learning decision trees. The cost-dependent information measure ensures the selection of the (local) next best, that is, cost-minimizing, discriminating attribute in the sequential construction of the classification trees. This is shown to be a cost-dependent generalization of the classical information measure introduced by Shannon, which only depends on classical probabilities. It is therefore of general importance and extends classic information theory, knowledge processing, and cognitive science, since subjective evaluations of decision alternatives can be included in entropy and the transferred information. Decision trees can then be viewed as cost-minimizing decoders for class symbols emitted by a source and coded by feature vectors. Experiments with two artificial datasets and one application example show that this approach is more accurate than a method which uses class dependent costs given by experts a priori.

show abstract

Section: Comparison With Other Algorithms Of Classification Learningmentioning

confidence: 99%

Section: Comparison With Other Algorithms Of Classification Learningmentioning

confidence: 99%

Section: Algorithmmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A New Information Measure Based on Example-Dependent Misclassification Costs and Its Application in Decision Tree Learning

Wysotzki

Geibel

2009

Advances in Artificial Intelligence

Self Cite

View full text Add to dashboard Cite

show abstract

“…Figure 6 shows typical resulting trajectories from the space-indexed controller, the pure time-indexed controller, and the time-indexed controller with re-indexing. Due to the stochasticity of in cost-sensitive SVM earning (Geibel et al, 2004). The algorithm finds the solution to the optimization problem: the real domain, the pure time-indexed approach performs very poorly.…”

Section: Autonomous Driving With Obstaclesmentioning

confidence: 99%

Space-indexed dynamic programming

Kolter

Coates

et al. 2008

Proceedings of the 25th International Conference on Machine Learning - ICML '08

View full text Add to dashboard Cite

We consider the task of learning to accurately follow a trajectory in a vehicle such as a car or helicopter. A number of dynamic programming algorithms such as Differential Dynamic Programming (DDP) and Policy Search by Dynamic Programming (PSDP), can efficiently compute non-stationary policies for these tasks -such policies in general are well-suited to trajectory following since they can easily generate different control actions at different times in order to follow the trajectory. However, a weakness of these algorithms is that their policies are timeindexed, in that they apply different policies depending on the current time. This is problematic since 1) the current time may not correspond well to where we are along the trajectory and 2) the uncertainty over states can prevent these algorithms from finding any good policies at all. In this paper we propose a method for space-indexed dynamic programming that overcomes both these difficulties. We begin by showing how a dynamical system can be rewritten in terms of a spatial index variable (i.e., how far along the trajectory we are) rather than as a function of time. We then use these space-indexed dynamical systems to derive space-indexed version of the DDP and PSDP algorithms. Finally, we show that these algorithms perform well on a variety of control tasks, both in simulation and on real systems.

show abstract

Apportioned margin approach for cost sensitive large margin classifiers

Gottlieb

Kaufman

Kontorovich

2021

Ann Math Artif Intell

View full text Add to dashboard Cite

We consider the problem of cost sensitive multiclass classification, where we would like to increase the sensitivity of an important class at the expense of a less important one. We adopt an apportioned margin framework to address this problem, which enables an efficient margin shift between classes that share the same boundary. The decision boundary between all pairs of classes divides the margin between them in accordance with a given prioritization vector, which yields a tighter error bound for the important classes while also reducing the overall out-of-sample error. In addition to demonstrating an efficient implementation of our framework, we derive generalization bounds, demonstrate Fisher consistency, adapt the framework to Mercer's kernel and to neural networks, and report promising empirical results on all accounts.

show abstract

Perceptron and SVM learning with generalized cost models

Cited by 13 publications

References 13 publications

A New Information Measure Based on Example-Dependent Misclassification Costs and Its Application in Decision Tree Learning

A New Information Measure Based on Example-Dependent Misclassification Costs and Its Application in Decision Tree Learning

Space-indexed dynamic programming

Apportioned margin approach for cost sensitive large margin classifiers

Contact Info

Product

Resources

About