The Little Engine(s) That Could: Scaling Online Social Networks

Pujol, Josep M.; Erramilli, Vijay; Siganos, Georgos; Yang, Xiaoyuan; Laoutaris, Nikolaos; Chhabra, Parminder; Rodríguez, P.

doi:10.1109/tnet.2012.2188815

Cited by 104 publications

(173 citation statements)

References 23 publications

Supporting

Mentioning

171

Contrasting

Unclassified

Order By: Relevance

“…From an engineering perspective, understanding OSNs can enable the design of better networked systems. For example, an OSN provider may want to understand the social graph and user activity in order to improve user experience by optimizing the design of their datacenters and/or data storage on the cloud [3]; or by providing personalized services and ads. A network provider may also want to understand the traffic generated by activities of OSN users in order to design mechanisms, such as caching [4] and traffic engineering [5], to better serve that traffic.…”

Section: Introductionmentioning

confidence: 99%

Practical Recommendations on Crawling Online Social Networks

Gjoka

Kurant

Butts

et al. 2011

IEEE J. Select. Areas Commun.

218

200

View full text Add to dashboard Cite

Abstract-Our goal in this paper is to develop a practical framework for obtaining a uniform sample of users in an online social network (OSN) by crawling its social graph. Such a sample allows to estimate any user property and some topological properties as well. To this end, first, we consider and compare several candidate crawling techniques. Two approaches that can produce approximately uniform samples are the MetropolisHasting random walk (MHRW) and a re-weighted random walk (RWRW). Both have pros and cons, which we demonstrate through a comparison to each other as well as to the "ground truth." In contrast, using Breadth-First-Search (BFS) or an unadjusted Random Walk (RW) leads to substantially biased results. Second, and in addition to offline performance assessment, we introduce online formal convergence diagnostics to assess sample quality during the data collection process. We show how these diagnostics can be used to effectively determine when a random walk sample is of adequate size and quality. Third, as a case study, we apply the above methods to Facebook and we collect the first, to the best of our knowledge, representative sample of Facebook users. We make it publicly available and employ it to characterize several key properties of Facebook.

show abstract

Section: Introductionmentioning

confidence: 99%

Practical Recommendations on Crawling Online Social Networks

Gjoka

Kurant

Butts

et al. 2011

IEEE J. Select. Areas Commun.

218

200

View full text Add to dashboard Cite

show abstract

“…In [18], a Social Partitioning and Replication middleware-(SPAR) is proposed that explores the social network graph from user interaction, and then performs joint partitioning and replication to ensure local data semantics for the users. Similarly, in [19], temporal activity hypergraphs are used to model user interactions in social network, and then mincut clustering is used to minimise the impact of DTs with minimum load-imbalance.…”

Section: Related Workmentioning

confidence: 99%

Workload-aware incremental repartitioning of shared-nothing distributed databases for scalable OLTP applications

Kamal

Murshed

Buyya

2016

Future Generation Computer Systems

View full text Add to dashboard Cite

h i g h l i g h t s• We propose incremental repartitioning of distributed OLTP databases for high-scalability.• We model two incremental repartitioning algorithm and lookup mechanism.• We develop a unique transaction generation model for simulation.• We derive novel impact metrics for distributed transactions.• Simulation results indicate adaptability of the methods scalable OLTP applications. On-line Transaction Processing (OLTP) applications often rely on shared-nothing distributed databases that can sustain rapid growth in data volume. Distributed transactions (DTs) that involve data tuples from multiple geo-distributed servers can adversely impact the performance of such databases, especially when the transactions are short-lived and these require immediate responses. The k-way min-cut graph clustering based database repartitioning algorithms can be used to reduce the number of DTs with acceptable level of load balancing. Web applications, where DT profile changes over time due to dynamically varying workload patterns, frequent database repartitioning is needed to keep up with the change. This paper addresses this emerging challenge by introducing incremental repartitioning. In each repartitioning cycle, DT profile is learnt online and k-way min-cut clustering algorithm is applied on a special sub-graph representing all DTs as well as those non-DTs that have at least one tuple in a DT. The latter ensures that the min-cut algorithm minimally reintroduces new DTs from the non-DTs while maximally transforming existing DTs into non-DTs in the new partitioning. Potential load imbalance risk is mitigated by applying the graph clustering algorithm on the finer logical partitions instead of the servers and relying on random one-to-one cluster-to-partition mapping that naturally balances out loads. Inter-server datamigration due to repartitioning is kept in check with two special mappings favouring the current partition of majority tuples in a cluster-the many-to-one version minimising data migrations alone and the one-toone version reducing data migration without affecting load balancing. A distributed data lookup process, inspired by the roaming protocol in mobile networks, is introduced to efficiently handle data migration without affecting scalability. The effectiveness of the proposed framework is evaluated on realistic TPC-C workloads comprehensively using graph, hypergraph, and compressed hypergraph representations used in the literature. To compare the performance of any incremental repartitioning framework without any bias of the external min-cut algorithm due to graph size variations, a transaction generation model is developed that can maintain a target number of unique transactions in any arbitrary observation window, irrespective of new transaction arrival rate. The overall impact of DTs at any instance is estimated from the exponential moving average of the recurrence period of unique transactions to avoid transient fluctuations. The effectiveness and adaptability of the proposed incremental repartition...

show abstract

“…NetTube [5] is a peer-assisted VoD system that leverages the content graph of UGC videos through social-aware pre-fetching and overlays to optimize swarming and decrease start-up delays. Finally, SPAR [12] is a social partitioning and replication system that achieves one-hop replication of user profiles in social networks.…”

Section: Related Workmentioning

confidence: 99%

Content and geographical locality in user-generated content sharing systems

Huguenin

Kermarrec

Kloudas

et al. 2012

Proceedings of the 22nd International Workshop on Network and Operating System Support for Digital Audio and Video

View full text Add to dashboard Cite

User Generated Content (UGC), such as YouTube videos, accounts for a substantial fraction of the Internet traffic. To optimize their performance, UGC services usually rely on both proactive and reactive approaches that exploit spatial and temporal locality in access patterns. Alternative types of locality are also relevant and hardly ever considered together. In this paper, we show on a large (more than 650,000 videos) YouTube dataset that content locality (induced by the related videos feature) and geographic locality, are in fact correlated. More specifically, we show how the geographic view distribution of a video can be inferred to a large extent from that of its related videos. We leverage these findings to propose a UGC storage system that proactively places videos close to the expected requests. Compared to a caching-based solution, our system decreases by 16% the number of requests served from a different country than that of the requesting user, and even in this case, the distance between the user and the server is 29% shorter on average.

show abstract

The Little Engine(s) That Could: Scaling Online Social Networks

Cited by 104 publications

References 23 publications

Practical Recommendations on Crawling Online Social Networks

Practical Recommendations on Crawling Online Social Networks

Workload-aware incremental repartitioning of shared-nothing distributed databases for scalable OLTP applications

Content and geographical locality in user-generated content sharing systems

Contact Info

Product

Resources

About