A primal-dual perspective of online learning algorithms

Shalev‐Shwartz, Shai; Singer, Yoram

doi:10.1007/s10994-007-5014-x

Cited by 141 publications

(59 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…FTRL dates back to the potential-based forecaster in [2, Chapter 11] and its theory was developed in [28]. The name Follow The Regularized Leader comes from [16].…”

Section: Previous Resultsmentioning

confidence: 99%

Scale-free online learning

Orabona

Pál

2018

Theoretical Computer Science

View full text Add to dashboard Cite

We design and analyze algorithms for online linear optimization that have optimal regret and at the same time do not need to know any upper or lower bounds on the norm of the loss vectors. Our algorithms are instances of the Follow the Regularized Leader (FTRL) and Mirror Descent (MD) meta-algorithms. We achieve adaptiveness to the norms of the loss vectors by scale invariance, i.e., our algorithms make exactly the same decisions if the sequence of loss vectors is multiplied by any positive constant. The algorithm based on FTRL works for any decision set, bounded or unbounded. For unbounded decisions sets, this is the first adaptive algorithm for online linear optimization with a non-vacuous regret bound. In contrast, we show lower bounds on scale-free algorithms based on MD on unbounded domains.

show abstract

“…FTRL dates back to the potential-based forecaster in [2, Chapter 11] and its theory was developed in [28]. The name Follow The Regularized Leader comes from [16].…”

Section: Previous Resultsmentioning

confidence: 99%

Scale-free online learning

Orabona

Pál

2018

Theoretical Computer Science

View full text Add to dashboard Cite

show abstract

“…A somewhat similar concept was re-discovered in online learning by Herbster and Warmuth [20], Grove, Littlestone, and Schuurmans [15], Kivinen and Warmuth [25] under the name of potential-based gradient descent, see [10,Chapter 11]. Recently, these ideas have been flourishing, see for instance ShalevSchwartz [33], Rakhlin [30], Hazan [17], and Bubeck [7]. Our main theorem (Theorem 2) allows one to recover almost all known regret bounds for online combinatorial optimization.…”

Section: Contribution and Contents Of The Papermentioning

confidence: 88%

Regret in Online Combinatorial Optimization

2014

View full text Add to dashboard Cite

We address online linear optimization problems when the possible actions of the decision maker are represented by binary vectors. The regret of the decision maker is the difference between her realized loss and the minimal loss she would have achieved by picking, in hindsight, the best possible action. Our goal is to understand the magnitude of the best possible (minimax) regret. We study the problem under three different assumptions for the feedback the decision maker receives: full information, and the partial information models of the socalled "semi-bandit" and "bandit" problems. In the full information case we show that the standard exponentially weighted average forecaster is a provably suboptimal strategy. For the semi-bandit model, by combining the Mirror Descent algorithm and the INF (Implicitely Normalized Forecaster) strategy, we are able to prove the first optimal bounds. Finally, in the bandit case we discuss existing results in light of a new lower bound, and suggest a conjecture on the optimal regret in that case.

show abstract

“…Indeed, Weighted Majority is an example of broader class of algorithms collectively known as Follow the Regularized Leader (FTRL) algorithms [34,35,58]. The FTRL template can be applied to a wide class of learning problems that fall under a general framework commonly known as online convex optimization [62].…”

Section: Online Learning and Regret-minimizing Algorithmsmentioning

confidence: 99%

Efficient Market Making via Convex Optimization, and a Connection to Online Learning

Abernethy

Chen

Vaughan

2013

ACM Trans. Econ. Comput.

103

View full text Add to dashboard Cite

We propose a general framework for the design of securities markets over combinatorial or infinite state or outcome spaces. The framework enables the design of computationally efficient markets tailored to an arbitrary, yet relatively small, space of securities with bounded payoff. We prove that any market satisfying a set of intuitive conditions must price securities via a convex cost function, which is constructed via conjugate duality. Rather than deal with an exponentially large or infinite outcome space directly, our framework only requires optimization over a convex hull. By reducing the problem of automated market making to convex optimization, where many efficient algorithms exist, we arrive at a range of new polynomial-time pricing mechanisms for various problems. We demonstrate the advantages of this framework with the design of some particular markets. We also show that by relaxing the convex hull we can gain computational tractability without compromising the market institution's bounded budget. Although our framework was designed with the goal of deriving efficient automated market makers for markets with very large outcome spaces, this framework also provides new insights into the relationship between market design and machine learning, and into the complete market setting. Using our framework, we illustrate the mathematical parallels between cost function based markets and online learning and establish a correspondence between cost function based markets and market scoring rules for complete markets.

show abstract

A primal-dual perspective of online learning algorithms

Cited by 141 publications

References 21 publications

Scale-free online learning

Scale-free online learning

Regret in Online Combinatorial Optimization

Efficient Market Making via Convex Optimization, and a Connection to Online Learning

Contact Info

Product

Resources

About