Fast Implementation of ℓ ₁ Regularized Learning Algorithms Using Gradient Descent Methods

Abstract

With the advent of high-throughput technologies, ℓ 1 regularized learning algorithms have attracted much attention recently. Dozens of algorithms have been proposed for fast implementation, using various advanced optimization techniques. In this paper, we demonstrate that ℓ 1 regularized learning problems can be easily solved by using gradient-descent techniques. The basic idea is to transform a convex optimization problem with a non-differentiable objective function into an unconstrained non-convex problem, upon which, via gradient descent, reaching a globally optimum solution is guaranteed. We present detailed implementation of the algorithm using ℓ 1 regularized logistic regression as a particular application. We conduct large-scale experiments to compare the new approach with other stateof-the-art algorithms on eight medium and large-scale problems. We demonstrate that our algorithm, though simple, performs similarly or even better than other advanced algorithms in terms of computational efficiency and memory usage.

Advanced computational algorithms for microbial community analysis using massive 16S rRNA sequence data

“…In this situation, special care must be taken to avoid overfitting problems. A commonly used practice is to select a small feature subset so that the performance of a learning algorithm is optimized ( 21–23 ). For the purpose of this article, we used regularized logistical regression to perform feature selection and classification simultaneously ( 23 ).…”

Cancer progression modeling using static sample data

“…The two steps were iterated until convergence. We used our recently developed gradient-descent-based algorithm to solve the above optimization problem efficiently [26]. By using the fixed-point theory [27], it can be proved that the algorithm converges to a unique solution regardless of the initial weights if the kernel width is properly selected.…”

A comparative study of improvements Pre-filter methods bring on feature selection using microarray data

“…We used L1 logistic regression to perform the feature selection procedures due to its ability to dispose the high dimensional data [ 40 ]. The model describes were as follows:…”