Experimental Evaluation of Train and Test Split Strategies in Link Prediction

Abstract

In link prediction, the goal is to predict which links will appear in the future of an evolving network. To estimate the performance of these models in a supervised machine learning model, disjoint and independent train and test sets are needed. However, objects in a realworld network are inherently related to each other. Therefore, it is far from trivial to separate candidate links into these disjoint sets.Here we characterize and empirically investigate the two dominant approaches from the literature for creating separate train and test sets in link prediction, referred to as random and temporal splits. Comparing the performance of these two approaches on several large temporal network datasets, we find evidence that random splits may result in too optimistic results, whereas a temporal split may give a more fair and realistic indication of performance. Results appear robust to the selection of temporal intervals. These findings will be of interest to researchers that employ link prediction or other machine learning tasks in networks.

Supervised temporal link prediction in large-scale real-world networks

“…Many different classification algorithms are known to work well in link prediction. Commonly used classifiers include logistic regression (Potgieter et al 2007;O'Madadhain et al 2005), support vector machines (Al Hasan et al 2006;Öczan A, Öğüdücü 2017), k-nearest neighbours (Al Hasan et al 2006;Bütün et al 2018Bütün et al , 2016, and random forests (Öczan A, Öğüdücü 2017;Bütün et al 2016Bütün et al , 2018Ghasemian et al 2020;de Bruin et al 2021de Bruin et al , 2020. We report performances using the logistic regression classifier.…”

“…A common approach in temporal link prediction is to employ a supervised machine learning model that utilises multiple features to classify which links are missing or, in case of temporal link prediction, will appear in the future (de Bruin et al 2021). Features are typically computed for every pair of nodes that is not (yet) connected, based on the topology of the network (Kumar et al 2020).…”

Link Prediction in Dynamic Networks Based on the Selection of Similarity Criteria and Machine Learning

“…These models struggle with over-fitting in link prediction because of the abundance of features. In other words, the designed machine learning model performs well on educational data but less well on experimental data [6]. In homogeneous networks, there is also the issue of an unbalanced data set.…”