Overcoming Bias in Community Detection Evaluation

Abstract

Community detection is a key task to further understand the function and the structure of complex networks. Therefore, a strategy used to assess this task must be able to avoid biased and incorrect results that might invalidate further analyses or applications that rely on such communities. Two widely used strategies to assess this task are generally known as structural and functional. The structural strategy basically consists in detecting and assessing such communities by using multiple methods and structural metrics. On the other hand, the functional strategy might be used when ground truth data are available to assess the detected communities. However, the evaluation of communities based on such strategies is usually done in experimental configurations that are largely susceptible to biases, a situation that is inherent to algorithms, metrics and network data used in this task. Furthermore, such strategies are not systematically combined in a way that allows for the identification and mitigation of bias in the algorithms, metrics or network data to converge into more consistent results. In this context, the main contribution of this article is an approach that supports a robust quality evaluation when detecting communities in real-world networks. In our approach, we measure the quality of a community by applying the structural and functional strategies, and the combination of both, to obtain different pieces of evidence. Then, we consider the divergences and the consensus among the pieces of evidence to identify and overcome possible sources of bias in community detection algorithms, evaluation metrics, and network data. Experiments conducted with several real and synthetic networks provided results that show the effectiveness of our approach to obtain more consistent conclusions about the quality of the detected communities.

INTELIGÊNCIA ARTIFICIAL NA EDUCAÇÃO: aplicações do aprendizado de máquina para apoiar a aprendizagem adaptativa

“…Em seguida, é preciso modelar um problema educacional real para ser solucionado pela tarefa escolhida. Note que, podem existir outros aspectos que devem ser considerados na aplicação de uma tarefa de IA, como a escolha do melhor algoritmo dentre os múltiplos que se diferenciam em eficiência e eficácia (Leão et al, 2020). Apesar disso, deixamos para aprofundar sobre esses aspectos nos materiais complementares e na continuidade deste trabalho, por exigir dos profissionais da educação conhecimentos muito específicos da área da computação.…”

Community detection in the human connectome: Method types, differences and their impact on inference

“…However, there is currently no true standard or systematic approach for method selection. In other fields, method dependence of modularity estimates has been documented, and has been shown to limit the interpretability, reproducibility, and/or relevance of findings, as well as scientific inference (Laender et al, 2020; Leskovec et al, 2010). In the context of the brain, to date, there are limited systematic assessments and comparisons of these methods as a function of network topology, particularly in the developing brain, which is both highly heterogeneous and undergoes profound age‐related topological changes.…”