Large margin classification using the perceptron algorithm

Freund, Yoav; Schapire, Robert E.

doi:10.1145/279943.279985

Cited by 603 publications

(608 citation statements)

References 11 publications

Supporting

Mentioning

589

Contrasting

Unclassified

Order By: Relevance

“…However such ad-hoc procedures of forcing convergence lead to bias in the final parameters. In the oscillatory case, one can choose any of the parameter selection heuristics commonly used in perceptron learning where convergence is also not guaranteed, e.g., the voted perceptron [14] [13]. In this work we simply used majority vote parameter setting, i.e., the parameters for which the training error was minimum.…”

Section: Experimental Observations: Parameter Learningmentioning

confidence: 99%

Exploiting Inference for Approximate Parameter Learning in Discriminative Fields: An Empirical Study

Kumar

August

Hebert

2005

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Estimation of parameters of random field models from labeled training data is crucial for their good performance in many image analysis applications. In this paper, we present an approach for approximate maximum likelihood parameter learning in discriminative field models, which is based on approximating true expectations with simple piecewise constant functions constructed using inference techniques. Gradient ascent with these updates exhibits compelling limit cycle behavior which is tied closely to the number of errors made during inference. The performance of various approximations was evaluated with different inference techniques showing that the learned parameters lead to good classification performance so long as the method used for approximating the gradient is consistent with the inference mechanism. The proposed approach is general enough to be used for the training of, e.g., smoothing parameters of conventional Markov Random Fields (MRFs).

show abstract

Section: Experimental Observations: Parameter Learningmentioning

confidence: 99%

Exploiting Inference for Approximate Parameter Learning in Discriminative Fields: An Empirical Study

Kumar

August

Hebert

2005

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…We compared the performances of our two-stage MB classifier with those of four widely used classifiers: a naïve Bayes classifier based on the multivariate Bernoulli distribution with Laplace prior for unseen words, discussed in Nigam et al [21], a support vector machine (SVM) classifier, discussed by Joachims [22], an implementation of the voted Perceptron, discussed in Freund and Schapire [23], and a maximum entropy conditional random field learner, introduced by Lafferty et al [24].…”

Section: Results and Analysismentioning

confidence: 99%

Markov Blankets and Meta-heuristics Search: Sentiment Extraction from Unstructured Texts

Airoldi

Bai

Padman³

2006

Advances in Web Mining and Web Usage Analysis

View full text Add to dashboard Cite

Abstract. Extracting sentiments from unstructured text has emerged as an important problem in many disciplines. An accurate method would enable us, for example, to mine online opinions from the Internet and learn customers' preferences for economic or marketing research, or for leveraging a strategic advantage. In this paper, we propose a two-stage Bayesian algorithm that is able to capture the dependencies among words, and, at the same time, finds a vocabulary that is efficient for the purpose of extracting sentiments. Experimental results on online movie reviews and online news show that our algorithm is able to select a parsimonious feature set with substantially fewer predictor variables than in the full data set and leads to better predictions about sentiment orientations than several state-of-the-art machine learning methods. Our findings suggest that sentiments are captured by conditional dependence relations among words, rather than by keywords or high-frequency words.

show abstract

“…the minimal distance from any instance to the separating hyperplane. Freund and Schapire (1998) generalized this result to the inseparable case. The maximum-margin algorithm uses quadratic programming to find the weight vector that classifies all the training data correctly and maximizes the margin.…”

Section: Introductionmentioning

confidence: 87%

Untitled

Long

2002

Machine Learning

132

View full text Add to dashboard Cite

Abstract. We describe a new incremental algorithm for training linear threshold functions: the Relaxed Online Maximum Margin Algorithm, or ROMMA. ROMMA can be viewed as an approximation to the algorithm that repeatedly chooses the hyperplane that classifies previously seen examples correctly with the maximum margin. It is known that such a maximum-margin hypothesis can be computed by minimizing the length of the weight vector subject to a number of linear constraints. ROMMA works by maintaining a relatively simple relaxation of these constraints that can be efficiently updated. We prove a mistake bound for ROMMA that is the same as that proved for the perceptron algorithm. Our analysis implies that the maximum-margin algorithm also satisfies this mistake bound; this is the first worst-case performance guarantee for this algorithm. We describe some experiments using ROMMA and a variant that updates its hypothesis more aggressively as batch algorithms to recognize handwritten digits. The computational complexity and simplicity of these algorithms is similar to that of perceptron algorithm, but their generalization is much better. We show that a batch algorithm based on aggressive ROMMA converges to the fixed threshold SVM hypothesis.

show abstract

Large margin classification using the perceptron algorithm

Cited by 603 publications

References 11 publications

Exploiting Inference for Approximate Parameter Learning in Discriminative Fields: An Empirical Study

Exploiting Inference for Approximate Parameter Learning in Discriminative Fields: An Empirical Study

Markov Blankets and Meta-heuristics Search: Sentiment Extraction from Unstructured Texts

Untitled

Contact Info

Product

Resources

About