Dual coordinate descent methods for logistic regression and maximum entropy models

Abstract

Most optimization methods for logistic regression or maximum entropy solve the primal problem. They range from iterative scaling, coordinate descent, quasi-Newton, and truncated Newton. Less efforts have been made to solve the dual problem. In contrast, for linear support vector machines (SVM), methods have been shown to be very effective for solving the dual problem. In this paper, we apply coordinate descent methods to solve the dual form of logistic regression and maximum entropy. Interestingly, many details are different from the situation in linear SVM. We carefully study the theoretical convergence as well as numerical issues. The proposed method is shown to be faster than most state of the art methods for training logistic regression and maximum entropy.

Machine Learning Approach to Classification of Childhood Disabilities for Special Education

“…In this research, we picked popular classification algorithms as classifiers covering various types, i.e., Logistic Regression [8], Support Vector Machine [9], Random Forest [10], Gradient Boosting [11] and Nearest Neighbors [12]. In this study, all the algorithms are implemented using Scikit-learn, a powerful machine learning library in Python [13].…”

Learning to Interpret and Describe Abstract Scenes

“…Examples of scenes, their descriptions, and corresponding templates are shown in Table 3 (template nsubj,aux,verb,det,dobj is the most frequent in the data). We train a logistic regression classifier (Yu et al, 2011) on scene-template pairs, and learn to assign a template for a new unseen scene. The "templatepredictor" uses variety of features based on the alignment between clip-art objects and POS-tags as well as objects and dependency roles.…”

Large-scale Reordering Model for Statistical Machine Translation using Dual Multinomial Logistic Regression

“…This is because the objective function (9) is not well defined at α ik = 0 due to the logarithm appearance. Finally, the optimal dual variables are achieved when the following condition is satisfied for all examples (Yu et al, 2011):…”

“…Similarly, Yu et al (2011) proposed a two-level dual coordinate descent method for maximum entropy classifier.…”