Learning Capability of Relaxed Greedy Algorithms

Abstract

In the practice of machine learning, one often encounters problems in which noisy data are abundant while the learning targets are imprecise and elusive. To these challenges, most of the traditional learning algorithms employ hypothesis spaces of large capacity. This has inevitably led to high computational burdens and caused considerable machine sluggishness. Utilizing greedy algorithms in this kind of learning environment has greatly improved machine performance. The best existing learning rate of various greedy algorithms is proved to achieve the order of (m/log m)(-1/2), where m is the sample size. In this paper, we provide a relaxed greedy algorithm and study its learning capability. We prove that the learning rate of the new relaxed greedy algorithm is faster than the order m(-1/2). Unlike many other greedy algorithms, which are often indecisive issuing a stopping order to the iteration process, our algorithm has a clearly established stopping criteria.

Approximation Properties of the Vector Weak Rescaled Pure Greedy Algorithm

“…They defined the Rescaled Pure Greedy Learning Algorithm (RPGLA) and studied its efficiency. They showed that the computational complexity of the RPGLA is less than the Orthogonal Greedy Learning Algorithm (OGLA) [37] and Relaxed Greedy Learning Algorithm (RGLA) [38]. When the kernel is infinitely smooth, the learning rate can be arbitrarily close to the best rate O(m −1 ) under a mild assumption of the regression function.…”

The distributed economic dispatch of smart grid based on deep reinforcement learning

“…The balance of exploration and exploitation is achieved by selecting actions according to the idea of ε-greedy algorithm [21]. Specifically, the action with the highest value can be obtained in the selected dispatch strategy π(s i,t ) with probability 1-ε, and start-stop a i,t of the unit i is randomly selected with probability ε.…”

Design methodologies for robust and low-overhead asynchronous quasi-delay-insensitive digital systems

“…The search space in the experiment herein is hence ~2 400 . Due to the gargantuan magnitude of the search space, some search algorithms, such as genetic algorithm [101][102][103][104][105][106], greedy algorithm [107][108][109][110][111][112], and dynamic programming [113][114][115][116][117], should be employed to search for the optimal solution.…”