Synthetic Test Data Generation for Hierarchical Graph Clustering Methods

Abstract

Abstract. Recent achievements in graph-based clustering algorithms revealed the need for large-scale test data sets. This paper introduces a procedure that can provide synthetic but realistic test data to the hierarchical Markov clustering algorithm. Being created according to the structure and properties of the SCOP95 protein sequence data set, the synthetic data act as a collection of proteins organized in a four-level hierarchy and a similarity matrix containing pairwise similarity values of the proteins. An ultimate high-speed TRIBE-MCL algorithm was employed to validate the synthetic data. Generated data sets have a healthy amount of variability due to the randomness in the processing, and are suitable for testing graph-based clustering algorithms on large-scale data.

Recent Advances in Improving the Memory Efficiency of the TRIBE MCL Algorithm

“…The proposed method underwent a series of benchmark tests using synthetic test data sets of sizes ranging from 10 thousand to one million items. Data sets were generated using the method indicated in [15]. For each data size, 21 instances were created and the one with median density was chosen for the test.…”

“…This change can significantly upgrade the size of processable data sets, and may also improve processing speed. The proposed method will be validated using large synthetic protein data sets derived from the SCOP95 database [2,10,14] using our method presented in [15].…”