PolderCast: Fast, Robust, and Scalable Architecture for P2P Topic-Based Pub/Sub

Abstract

Abstract. We propose PolderCast, a P2P topic-based Pub/Sub system that is (a) fault-tolerant and robust, (b) scalable w.r.t the number of nodes interested in a topic and number of topics that nodes are interested in, and (c) fast in terms of dissemination latency while (d) attaining a low communication overhead. This combination of properties is provided by an implementation that blends deterministic propagation over maintained rings with probabilistic dissemination following a limited number of random shortcuts. The rings are constructed and maintained using gossiping techniques. The random shortcuts are provided by two distinct peer-sampling services: Cyclon generates purely random links while Vicinity produces interest-induced random links. We analyze PolderCast and survey it in the context of existing approaches. We evaluate PolderCast experimentally using real-world workloads from Twitter and Facebook traces. We use widely renowned Scribe [5] as a baseline in a number of experiments. Robustness with respect to node churn is evaluated through traces from the Skype superpeer network. We show that the experimental results corroborate all of the above properties in settings of up to 10K nodes, 10K topics, and 5K topics per-node.

Designing Overlay Networks for Handling Exhaust Data in a Distributed Topic-based Pub/Sub Architecture

“…The property of relay-free [10], also called Topic-connected Overlay, is primarily discussed in studies based on unstructured overlay networks [11]. The definition is as follows:…”

“…For overcoming such inefficiencies, we first clarify the requirements of overlay networks for improving the adaptivity and define a desirable property called "strong relay-free" by expanding relay-free, which is mainly known in studies based on unstructured overlay networks [10], [11]. In the strong relay-free property, publishers and subscribers compose connected subgraphs respectively, and these subgraphs are also connected to each other.…”

Distributed Ranked Data Dissemination in Social Networks

“…All these subscriptions are topic-based: Subscribers are interested in the totality of publications posted by a user. From the original datasets, we extracted a sample set contained subscriptions for 1000 users, using the technique employed in [27]. The samples retain the original properties of the original datasets, specifically with regards to the relative popularity of the topics.…”

“…Indeed, a rising trend is to establish a decentralized architecture in social networks such as Diaspora. The pub/sub model, known for its scalability and decoupled nature, then becomes a logical candidate for the dissemination substrate of distributed social networks, and pub/sub designs for social networks have recently surfaced [27,30]. Therefore, our goal is to provide a solution for distributed ranked data dissemination for pub/sub systems within the context of social networks.…”

Reduction of Unnecessarily Ordered Event Messages in Peer-to-Peer Model of Topic-Based Publish/Subscribe Systems

“…sequence of events [19]. Event-driven distributed systems like publish/subscribe (PS) systems [7], [9], [20], [21], [22] are developed and used in various types of distributed applications like Google alert [8]. We discuss a peer-to-peer PS (P2PPS) model [15], [16] which is composed of peer processes (peers) [23], [24] where each peer can play both subscriber and publisher roles.…”