A Novel Clustering Methodology Based on Modularity Optimisation for Detecting Authorship Affinities in Shakespearean Era Plays

Abstract

In this study we propose a novel, unsupervised clustering methodology for analyzing large datasets. This new, efficient methodology converts the general clustering problem into the community detection problem in graph by using the Jensen-Shannon distance, a dissimilarity measure originating in Information Theory. Moreover, we use graph theoretic concepts for the generation and analysis of proximity graphs. Our methodology is based on a newly proposed memetic algorithm (iMA-Net) for discovering clusters of data elements by maximizing the modularity function in proximity graphs of literary works. To test the effectiveness of this general methodology, we apply it to a text corpus dataset, which contains frequencies of approximately 55,114 unique words across all 168 written in the Shakespearean era (16th and 17th centuries), to analyze and detect clusters of similar plays. Experimental results and comparison with state-of-the-art clustering methods demonstrate the remarkable performance of our new method for identifying high quality clusters which reflect the commonalities in the literary style of the plays.

Clustering algorithms: A comparative approach

“…Clustering data is a complex task involving the choice between many different methods, parameters and performance metrics, with implications in many real-world problems [63,[88][89][90][91][92][93]. Consequently, the analysis of the advantages and pitfalls of clustering algorithms is also a difficult task that has been received much attention.…”

On The Use of Optimization for Mining Clusters in Networks

“…In addition, these approaches focus on optimizing the modularity which is considered the most common and prominent category of modules discovery approaches, has been employed successfully for dealing with numerous applications, and is still utilized in so many recent research papers as it has become the main objective function in these modern approaches [12], [14]. However, it is not the ideal measure to be optimized.…”

“…In addition, these algorithms have been utilized for handling the clustering problem after transforming it into a community detection problem via the utilization of a dissimilarity metric [14]. Similarly, they were utilized by Bracamonte et al [15] to cluster the tags resulted from the multimedia search.…”

“…The aim of discovering community structure in networks is to get an adequate categorization where the links to the nodes in an interesting community are dense, but they are sparse to the nodes outside of this community in the case of nonoverlapping groups [3] and also discovering the vertices which belong to multiple groups (known as overlapping vertices) [17] since items have usually distinctive roles and hence are members of multiple modules as in the real world, i.e., nodes are often shared between different groups in the graph representing them [9], [14], [17]. Being a member in numerous communities is highly prevalent in the real-world networks.…”

“…Despite various techniques have been utilized recently for discovering the densely connected clusters in networks, this crucial problem in various disciplines is still considered an open and a hard problem and has not been tackled satisfactorily since the networks have usually complex nature [14], [20]. More particularly, in huge networks, discovering successfully their module structure is still considered a great data mining problem [21].…”