A Java-Based Distributed Approach for Generating Large-Scale Social Network Graphs

Abstract

Distributed systems and applications require large amounts of resources in terms of memory and computing power and are becoming a standard for large businesses and enterprises [12] within and outside the domain of Computer Science. A very important topic for distributed applications is Big Data management and more specifically the generation of large-scale social networks graphs where the number of nodes reaches very large numbers. Analysis of such networks is of importance in many areas, e.g., data mining, network sciences, physics, and social sciences [3]. The need for efficient and scalable methods of network generation is frequently mentioned in the literature [8], particularly for the preferential attachment process [1,13,14]. Barabasi-Albert model, which is based on preferential attachment (PA) [4], is one of the most commonly used models to produce artificial networks, because of its explanatory power, conceptual simplicity, and interesting mathematical properties [13]. Nevertheless the large number of nodes in such graphs may not fit in the memory on one machine. The need for efficient solutions which provide scalability also requires more computational resources as well as implementation considerations. As such, distribution and synchronization are two main challenges. In this chapter, we investigate as a case study a distributed solution for PA-based graph generation which avoids low level synchronization management, thanks to the notion of cooperative scheduling and futures.

Distributed Network Generation Based on Preferential Attachment in ABS

“…The choice of Haskell was made mainly for two reasons: the ABS-Haskell backend seems to be currently the fastest in terms of speed and memory use, attributed perhaps to the close match of the two languages in terms of language features: Haskell is also a high-level, statically-typed, purely functional language. Secondly, compared to the distributed implementation sketched in Java [16], the ABS-Haskell runtime utilizes the support of Haskell's lightweight threads and first-class continuations to efficiently implement multicore-enabled cooperative scheduling; Java does not have built-in language support for algebraic datatypes, continuations and its system OS threads (heavyweight) makes it a less ideal candidate to implement cooperative scheduling in a straightforward manner. On the distributed side, layering our solution on top of Java RMI (Remote Method Invocation) framework was decided against for lack of built-in support for asynchronous remote method calls and superfluous features to our needs, such as code-transfer and fully-distributed garbage collection.…”

“…In this section, we present a high-level distributed solution for PA which is similar to the ones proposed for multicore architectures in [20] and distributed architectures in [15,16], in a sense that they adopt copy model introduced in [21] to represent the graph. To this aim, the description of the main data structure used to model the graph which represents the social network is given.…”

“…In [16] a high-level distributed model of the PA in ABS has been presented together with a high-level description of its possible implementation in Java. However, as we argue in Sect.…”