Rank minimization via online learning

Meka, Raghu; Jain, Prateek; Caramanis, Constantine; Dhillon, Inderjit S.

doi:10.1145/1390156.1390239

Cited by 57 publications

(51 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A similar result applies for semidefinite programming (based on an existing primal MWU-based SDP algorithm (Arora et al, 2005b)) as well as other optimizations for which the MWU applies, such as rank minimization (Meka et al, 2008), etc.…”

Section: Find Feasible X(t) Inmentioning

confidence: 70%

Efficient Protocols for Distributed Classification and Optimization

Daumé

Phillips

Saha

et al. 2012

Lecture Notes in Computer Science

View full text Add to dashboard Cite

In distributed learning, the goal is to perform a learning task over data distributed across multiple nodes with minimal (expensive) communication. Prior work (Daumé III et al., 2012) proposes a general model that bounds the communication required for learning classifiers while allowing for ε training error on linearly separable data adversarially distributed across nodes.In this work, we develop key improvements and extensions to this basic model. Our first result is a two-party multiplicative-weight-update based protocol that uses O(d 2 log 1/ε) words of communication to classify distributed data in arbitrary dimension d, ε-optimally. This readily extends to classification over k nodes with O(kd 2 log 1/ε) words of communication. Our proposed protocol is simple to implement and is considerably more efficient than baselines compared, as demonstrated by our empirical results.In addition, we illustrate general algorithm design paradigms for doing efficient learning over distributed data. We show how to solve fixed-dimensional and high dimensional linear programming efficiently in a distributed setting where constraints may be distributed across nodes. Since many learning problems can be viewed as convex optimization problems where constraints are generated by individual points, this models many typical distributed learning scenarios. Our techniques make use of a novel connection from multipass streaming, as well as adapting the multiplicative-weight-update framework more generally to a distributed setting. As a consequence, our methods extend to the wide range of problems solvable using these techniques.

show abstract

Section: Find Feasible X(t) Inmentioning

confidence: 70%

Efficient Protocols for Distributed Classification and Optimization

Daumé

Phillips

Saha

et al. 2012

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…Equation 7 restricts the representation such that it does not contain any part of the original vector itself. This problem turns out to be non-convex and NP-hard [7]. Nevertheless, recent developments in the optimization field have provided heuristics that are able to find relaxed solutions that are sparse [8].…”

Section: Interaction Motion As Sparse Subspace Separationmentioning

confidence: 99%

Recognizing Human-Object Interactions Using Sparse Subspace Clustering

Bogun

Ribeiro

2013

Computer Analysis of Images and Patterns

View full text Add to dashboard Cite

Abstract. In this paper, we approach the problem of recognizing humanobject interactions from video data. Using only motion trajectories as input, we propose an unsupervised framework for clustering and classifying videos of people interacting with objects. Our method is based on the concept of sparse subspace clustering, which has been recently applied to motion segmentation. Here, we show that human-object interactions can be seen as trajectories lying on a low-dimensional subspace, and which can in turn be recovered by subspace clustering. Experimental results, performed on a publicly available dataset, show that our approach is comparable to the state-of-the-art.

show abstract

“…It has since been used in boosting and in the generalized experts framework in machine learning [16], [23], [3]. And, this approach has been used to create an efficient algorithm for low-rank matrix approximation with guarantees [25]. To the best of our knowledge, our paper is the first to exploit these ideas to improve the running time for any MDP algorithms.…”

Section: Introductionmentioning

confidence: 99%

Efficient Algorithms for Budget-Constrained Markov Decision Processes

Caramanis

Dimitrov

Morton

2014

IEEE Trans. Automat. Contr.

Self Cite

View full text Add to dashboard Cite

Abstract-Discounted, discrete-time, discrete state-space, discrete action-space Markov decision processes (MDPs) form a classical topic in control, game theory, and learning, and as a result are widely applied, increasingly, in very large-scale applications. Many algorithms have been developed to solve large-scale MDPs. Algorithms based on value iteration are particularly popular, as they are more efficient than the generic linear programming approach, by an order of magnitude in the number of states of the MDP. Yet in the case of budget constrained MDPs, no more efficient algorithm than linear programming is known. The theoretically slower running times of linear programming may limit the scalability of constrained MDPs piratically; while, theoretically, it invites the question of whether the increase is somehow intrinsic. In this paper we show that it is not, and provide two algorithms for budgetconstrained MDPs that are as efficient as value iteration. Denoting the running time of value iteration by VI, and the magnitude of the input by U , for an MDP with m expected budget constraints our first algorithm runs in time O(poly(m, log U ) · VI). Given a pre-specified degree of precision, η, for satisfying the budget constraints, our second algorithm runs in time O(log m·poly(log U )· 1 η 2 ·VI), but may produce solutions that overutilize each of the m budgets by a multiplicative factor of 1 + η. In fact, one can substitute value iteration with any algorithm, possibly specially designed for a specific MDP, that solves the MDP quickly to achieve similar theoretical guarantees. Both algorithms restrict attention to constrained infinite-horizon MDPs under discounted costs.

show abstract

Rank minimization via online learning

Cited by 57 publications

References 18 publications

Efficient Protocols for Distributed Classification and Optimization

Efficient Protocols for Distributed Classification and Optimization

Recognizing Human-Object Interactions Using Sparse Subspace Clustering

Efficient Algorithms for Budget-Constrained Markov Decision Processes

Contact Info

Product

Resources

About