WHATSUP: A Decentralized Instant News Recommender

Boutet, Antoine; Frey, Davide; Guerraoui, Rachid; Jégou, Arnaud; Kermarrec, Anne-Marie

doi:10.1109/ipdps.2013.47

Cited by 42 publications

(47 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A large body of works have been proposed to construct KNN graphs in decentralized systems, with applications ranging from recommendation [4,14,19], to search [13], to news dissemination [6]. In such systems, nodes (e.g.…”

Section: Background: Decentralized K-nn Graph Constructionmentioning

confidence: 99%

See 1 more Smart Citation

Scalable Anti-KNN: Decentralized Computation of k-Furthest-Neighbor Graphs with HyFN

Bouget

Bromberg

Taı̈ani

et al. 2017

Distributed Applications and Interoperable Systems

View full text Add to dashboard Cite

International audienceThe decentralized construction of k-Furthest-Neighbor graphs has been little studied, although such structures can play a very useful role, for instance in a number of distributed resource allocation problems. In this paper we define KFN graphs; we propose HyFN, a generic peer-to-peer KFN construction algorithm, and thoroughly evaluate its behavior on a number of logical networks of varying sizes. 1 Motivation k-Nearest-Neighbor (KNN) graphs have found usage in a number of domains, including machine learning, recommenders, and search. Some applications do not however require the k closest nodes, but the k most dissimilar nodes, what we term the k-Furthest-Neighbor (KFN) graph. Virtual Machines (VMs) placement —i.e. the (re-)assignment of workloads in virtualised IT environments— is a good example of where KFN can be applied. The problem consists in finding an assignment of VMs on physical machines (PMs) that minimises some cost function(s) [27]. The problem has been described as one of the most complex and important for the IT industry [3], with large potential savings [20]. An important challenge is that a solution does not only consist in packing VMs onto PMs — it also requires to limit the amount of interferences between VMs hosted on the same PM [31]. Whatever technique is used (e.g. clustering [21]), interference aware VM placement algorithms need to identify complementary workloads — i.e. workloads that are dissimilar enough that the interferences between them are minimised. This is why the application of KFN graphs would make a lot of sense: identifying quickly complementary workloads (using KFN) to help placement algorithms would decrease the risks of interferences. The construction of KNN graphs in decentralized systems has been widely studied in the past [17, 30, 4, 14]. However, existing approaches typically assume a form of " likely transitivity " of similarity between nodes: if A is close to B, and B to C, then A is likely to be close to C. Unfortunately this property no longer holds when constructing KFN graphs. As a result, these approaches, as demonstrated in the remainder of the paper, are not working anymore when applied to this new problem

show abstract

Section: Background: Decentralized K-nn Graph Constructionmentioning

confidence: 99%

“…Similarly, when receiving a new neighborhood pushed to them, nodes update their local view with the new nodes they have just heard of (lines [6][7][8]. The intuition behind this greedy procedure is that if A is similar to B, and B to C, C is likely to be similar to A as well.…”

Section: Background: Decentralized K-nn Graph Constructionmentioning

confidence: 99%

Scalable Anti-KNN: Decentralized Computation of k-Furthest-Neighbor Graphs with HyFN

Bouget

Bromberg

Taı̈ani

et al. 2017

Distributed Applications and Interoperable Systems

View full text Add to dashboard Cite

show abstract

“…In such systems, each peer is connected to a subset of other peers in the network and periodically exchanges some information with one of its neighbors. While such protocols have been initially used to build uniform random topologies [19,21], they have also been applied in the context of several applications to cluster peers according to some specific metric (interest, overlap, etc) to build networks of arbitrary structure [18,24] or to support various applications such as query expansion [7], top-k queries [6] or news recommendation [9]. In such a system, the use of random nodes ensures that connectivity is maintained, each node is responsible to discover its KNN nodes by periodically exchanging neighborhood information with other peers.…”

Section: Related Workmentioning

confidence: 99%

Nearest Neighbors Graph Construction: Peer Sampling to the Rescue

2016

Self Cite

View full text Add to dashboard Cite

Abstract. In this paper, we propose an efficient KNN service, called KPS (KNN-Peer-Sampling). The KPS service can be used in various contexts e.g. recommendation systems, information retrieval and data mining. KPS borrows concepts from P2P gossip-based clustering protocols to provide a localized and efficient KNN computation in large-scale systems. KPS is a sampling-based iterative approach, combining randomness, to provide serendipity and avoid local minimum, and clustering, to ensure fast convergence. We compare KPS against the state of the art KNN centralized computation algorithm NNDescent, on multiple datasets. The experiments confirm the efficiency of KPS over NNDescent: KPS improves significantly on the computational cost while converging quickly to a close to optimal KNN graph. For instance, the cost, expressed in number of pairwise similarity computations, is reduced by ≈ 23% and ≈ 49% to construct high quality KNN graphs for Jester and MovieLens datasets, respectively. In addition, the randomized nature of KPS ensures eventual convergence, not always achieved with NNDescent.

show abstract

“…These metrics have been widely used in distributed systems where the user has a partial vision of the network [22,5] and also in graph-based solutions [16,3]. The main reason for their utilization is that they are computationally light and summarize relevant features users may have in common.…”

Section: Proposed Approachmentioning

confidence: 99%

Cheap and Cheerful: Trading Speed and Quality for Scalable Social-Recommenders

Kermarrec¹,

Taı̈ani²,

Tirado³

2015

Distributed Applications and Interoperable Systems

Self Cite

View full text Add to dashboard Cite

Abstract. Recommending appropriate content and users is a critical feature of on-line social networks. Computing accurate recommendations on very large datasets can however be particularly costly in terms of resources, even on modern parallel and distributed infrastructures. As a result, modern recommenders must generally trade-off quality and computational cost to reach a practical solution. This trade-off has however so far been largely left unexplored by the research community, making it difficult for practitioners to reach informed design decisions. In this paper, we investigate to which extent the additional computing costs of advanced recommendation techniques based on supervised classifiers can be balanced by the gains they bring in terms of quality. In particular, we compare these recommenders against their unsupervised counterparts, which offer lightweight and highly scalable alternatives. We propose a thorough evaluation comparing 11 classifiers against 7 lightweight recommenders on a real Twitter dataset. Additionally, we explore data grouping as a method to reduce computational costs in a distributed setting while improving recommendation quality. We demonstrate how classifiers trained using data grouping can reduce their computing time by 6 while improving recommendations up to 22% when compared with lightweight solutions.

show abstract

WHATSUP: A Decentralized Instant News Recommender

Cited by 42 publications

References 24 publications

Scalable Anti-KNN: Decentralized Computation of k-Furthest-Neighbor Graphs with HyFN

Scalable Anti-KNN: Decentralized Computation of k-Furthest-Neighbor Graphs with HyFN

Nearest Neighbors Graph Construction: Peer Sampling to the Rescue

Cheap and Cheerful: Trading Speed and Quality for Scalable Social-Recommenders

Contact Info

Product

Resources

About