Margin-Sparsity Trade-Off for the Set Covering Machine

Laviolette, François; Marchand, Mario; Shah, Mohak

doi:10.1007/11564096_23

Cited by 6 publications

(4 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is an extension of the works in Laviolette et al (2005Laviolette et al ( , 2006, Shah (2006). Basically, in order to incorporate the notion of margin, we use two approaches.…”

Section: Is It Worthwhile To Investigate If Classifiers With Improvedsupporting

confidence: 63%

Learning the set covering machine by bound minimization and margin-sparsity trade-off

et al. 2009

Self Cite

View full text Add to dashboard Cite

We investigate classifiers in the sample compression framework that can be specified by two distinct sources of information: a compression set and a message string of additional information. In the compression setting, a reconstruction function specifies a classifier when given this information. We examine how an efficient redistribution of this reconstruction information can lead to more general classifiers. In particular, we derive risk bounds that can provide an explicit control over the sparsity of the classifier and the magnitude of its separating margin and a capability to perform a margin-sparsity trade-off in favor of better classifiers. We show how an application to the set covering machine algorithm results in novel learning strategies. We also show that these risk bounds are tighter than their traditional counterparts such as VC-dimension and Rademacher complexity-based bounds that explicitly take into account the hypothesis class complexity. Finally, we show how these bounds are able to guide the model selection for the set covering machine algorithm enabling it to learn by bound minimization.

show abstract

“…This is an extension of the works in Laviolette et al (2005Laviolette et al ( , 2006, Shah (2006). Basically, in order to incorporate the notion of margin, we use two approaches.…”

Section: Is It Worthwhile To Investigate If Classifiers With Improvedsupporting

confidence: 63%

Learning the set covering machine by bound minimization and margin-sparsity trade-off

et al. 2009

Self Cite

View full text Add to dashboard Cite

show abstract

“…In that setting, given an a priori defined vector i of indices, one can use the examples of the training set that do not correspond to any index of i to bound the risk of the classifier defined by i (and the training set S). Moreover, provided a prior distribution is given on the set of all possible vector of indices, one can extend such a bound to a bound which is valid simultaneously for all classifiers that can be reconstructed [8].…”

Section: A Uniform Risk Bound For Active Learningmentioning

confidence: 99%

A Selective Sampling Strategy for Label Ranking

Amini¹,

Usunier²,

Laviolette³

et al. 2006

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

We propose a novel active learning strategy based on the compression framework of [9] for label ranking functions which, given an input instance, predict a total order over a predefined set of alternatives. Our approach is theoretically motivated by an extension to ranking and active learning of Kääriäinen's generalization bounds using unlabeled data [7], initially developed in the context of classification. The bounds we obtain suggest a selective sampling strategy provided that a sufficiently, yet reasonably large initial labeled dataset is provided. Experiments on Information Retrieval corpora from automatic text summarization and question/answering show that the proposed approach allows to substantially reduce the labeling effort in comparison to random and heuristic-based sampling strategies.

show abstract

“…This allows us to bound the generalisation error of the classifier using risk bounds based on data-depedent compression schemes. Considering using VS in a batch setting, from [14] we have the following:…”

Section: B Benchmark Datamentioning

confidence: 99%

Voted Spheres: An Online, Fast Approach to Large Scale Learning

Farran

Saunders

2009

2009 International Conference on Advanced Information Networking and Applications Workshops

View full text Add to dashboard Cite

In this paper, we introduce a novel, non-linear, fast, online algorithm for learning on large data sets. This algorithm, which we call Voted Spheres (VS) is a combination of hypersphere-fitting, and the idea of voting. The algorithm builds hyperspheres around points, with different hyperspheres belonging to different classes allowed to overlap. The advantages of the algorithm are that it is simple to implement, very efficient, and generalises well while being able to handle millions of data points. For the KDD intrusion detection data set consisting of 494, 020 data points, the linear version of the algorithm requires under a minute on a standard desktop PC and achieves state of the art performance.

show abstract

Margin-Sparsity Trade-Off for the Set Covering Machine

Abstract: Abstract. We propose a new learning algorithm for the set covering machine and a tight data-compression risk bound that the learner can use for choosing the appropriate tradeoff between the sparsity of a classifier and the magnitude of its separating margin.

Cited by 6 publications

References 8 publications

Learning the set covering machine by bound minimization and margin-sparsity trade-off

Learning the set covering machine by bound minimization and margin-sparsity trade-off

A Selective Sampling Strategy for Label Ranking

Voted Spheres: An Online, Fast Approach to Large Scale Learning

Contact Info

Product

Resources

About