Finding Statistically Significant Communities in Networks

Lancichinetti, Andrea; Radicchi, Filippo; Ramasco, José J.; Fortunato, Santo

doi:10.1371/journal.pone.0018961

Cited by 866 publications

(732 citation statements)

References 67 publications

Supporting

Mentioning

712

Contrasting

Unclassified

Order By: Relevance

“…The memory effects on community detection have interesting network-theoretical implications. Community-detection methods typically identify modules with stronger internal than external connections 42,43 or with relatively long flow persistence times 30,31 . A problem with these methods is that they tend to assign each node to a very limited number of modules, in contrast to the observation that real modules often show pervasive overlap [44][45][46] .…”

Section: First-order Markovmentioning

confidence: 99%

Memory in network flows and its effects on spreading dynamics and community detection

et al. 2014

Self Cite

View full text Add to dashboard Cite

Random walks on networks is the standard tool for modelling spreading processes in social and biological systems. This first-order Markov approach is used in conventional community detection, ranking and spreading analysis, although it ignores a potentially important feature of the dynamics: where flow moves to may depend on where it comes from. Here we analyse pathways from different systems, and although we only observe marginal consequences for disease spreading, we show that ignoring the effects of second-order Markov dynamics has important consequences for community detection, ranking and information spreading. For example, capturing dynamics with a second-order Markov model allows us to reveal actual travel patterns in air traffic and to uncover multidisciplinary journals in scientific communication. These findings were achieved only by using more available data and making no additional assumptions, and therefore suggest that accounting for higher-order memory in network flows can help us better understand how real systems are organized and function.

show abstract

Section: First-order Markovmentioning

confidence: 99%

Memory in network flows and its effects on spreading dynamics and community detection

et al. 2014

Self Cite

View full text Add to dashboard Cite

show abstract

“…We compared our algorithm to the best existing algorithms for detecting overlapping communities (2,8,9,(11)(12)(13)17). Each algorithm analyzes the (unlabeled) network and returns both the Fig.…”

Section: A Study Of Real and Synthetic Networkmentioning

confidence: 99%

“…The first is that many existing community detection algorithms assume that each node belongs to a single community (1,(3)(4)(5)(6)(7)(14)(15)(16). In real-world networks, each node will likely belong to multiple communities and its connections will reflect these multiple memberships (2,(8)(9)(10)(11)(12)(13)17). For example, in a large social network, a member may be connected to coworkers, friends from school, and neighbors.…”

mentioning

confidence: 99%

“…network analysis | Bayesian statistics | massive data C ommunity detection algorithms (1)(2)(3)(4)(5)(6)(7)(8)(9)(10)(11)(12)(13)(14)(15)(16)(17) analyze networks to find groups of densely connected nodes. These algorithms have become vital to data-driven methods for understanding and exploring network data such as social networks (4), citation networks (18), communication networks (19), and networks induced by scientific observation [e.g., gene regulation networks (20)].…”

mentioning

confidence: 99%

See 1 more Smart Citation

Efficient discovery of overlapping communities in massive networks

Gopalan

Blei

2013

Proc. Natl. Acad. Sci. U.S.A.

220

162

View full text Add to dashboard Cite

Detecting overlapping communities is essential to analyzing and exploring natural networks such as social networks, biological networks, and citation networks. However, most existing approaches do not scale to the size of networks that we regularly observe in the real world. In this paper, we develop a scalable approach to community detection that discovers overlapping communities in massive realworld networks. Our approach is based on a Bayesian model of networks that allows nodes to participate in multiple communities, and a corresponding algorithm that naturally interleaves subsampling from the network and updating an estimate of its communities. We demonstrate how we can discover the hidden community structure of several real-world networks, including 3.7 million US patents, 575,000 physics articles from the arXiv preprint server, and 875,000 connected Web pages from the Internet. Furthermore, we demonstrate on large simulated networks that our algorithm accurately discovers the true community structure. This paper opens the door to using sophisticated statistical models to analyze massive networks. Community detection is important for both exploring a network and predicting connections that are not yet observed. For example, by finding the communities in a large citation graph of scientific articles, we can make hypotheses about the fields and subfields that they contain. By finding communities in a large social network, we can more easily make predictions to individual members about who they might be friends with but are not yet connected to.In this paper, we develop an algorithm that discovers communities in modern real-world networks. The challenge is that real-world networks are massive-they can contain hundreds of thousands or even millions of nodes. We will examine a network of scientific articles that contains 575,000 articles, a network of connected Web pages that contains 875,000 pages, and a network of US patents that contains 3,700,000 patents. Most approaches to community detection cannot handle data at this scale.There are two fundamental difficulties to detecting communities in such networks. The first is that many existing community detection algorithms assume that each node belongs to a single community (1,(3)(4)(5)(6)(7)(14)(15)(16). In real-world networks, each node will likely belong to multiple communities and its connections will reflect these multiple memberships (2,(8)(9)(10)(11)(12)(13)17). For example, in a large social network, a member may be connected to coworkers, friends from school, and neighbors. We need algorithms that discover overlapping communities to capture the heterogeneity of each node's connections.The second difficulty is that existing algorithms are too slow. Many community detection algorithms iteratively analyze each pair of nodes, regardless of whether the nodes in the pair are connected in the network (5, 6, 10). Consequently, these algorithms run in time squared in the number of nodes, which makes analyzing massive networks computationally intractable. Other a...

show abstract

“…It has a high performance and is scalable to large networks [19]. -Louvain: This hierarchical clustering algorithm uses modularity as its objective function and maximizes it using multiple heuristics to detect the groups.…”

Section: Clustering Algorithmsmentioning

confidence: 99%

Friend Grouping Algorithms for Online Social Networks: Preference, Bias, and Implications

Eslami

Aleyasen

Moghaddam

et al. 2014

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Managing friendship relationships in social media is challenging due to the growing number of people in online social networks (OSNs). To deal with this challenge, OSNs' users may rely on manually grouping friends with personally meaningful labels. However, manual grouping can become burdensome when users have to create multiple groups for various purposes such as privacy control, selective sharing, and filtering of content. More recently, recommendation-based grouping tools such as Facebook smart lists have been proposed to address this concern. In these tools, users must verify every single friend suggestion. This can hinder users' adoption when creating large content sharing groups. In this paper, we proposed an automated friend grouping tool that applies three clustering algorithms on a Facebook friendship network to create groups of friends. Our goal was to uncover which algorithms were better suited for social network groupings and how these algorithms could be integrated into a grouping interface. In a series of semi-structured interviews, we asked people to evaluate and modify the groupings created by each algorithm in our interface. We observed an overwhelming consensus among the participants in preferring this automated grouping approach to existing recommendation-based techniques such as Facebook smart lists. We also discovered that the automation created a significant bias in the final modified groups. Finally, we found that existing group scoring metrics do not translate well to OSN groupings-new metrics are needed. Based on these findings, we conclude with several design recommendations to improve automated friend grouping approaches in OSNs.

show abstract

Finding Statistically Significant Communities in Networks

Cited by 866 publications

References 67 publications

Memory in network flows and its effects on spreading dynamics and community detection

Memory in network flows and its effects on spreading dynamics and community detection

Efficient discovery of overlapping communities in massive networks

Friend Grouping Algorithms for Online Social Networks: Preference, Bias, and Implications

Contact Info

Product

Resources

About