Evaluation of entity resolution approaches on real-world match problems

Abstract

Despite the huge amount of recent research efforts on entity resolution (matching) there has not yet been a comparative evaluation on the relative effectiveness and efficiency of alternate approaches. We therefore present such an evaluation of existing implementations on challenging real-world match tasks. We consider approaches both with and without using machine learning to find suitable parameterization and combination of similarity functions. In addition to approaches from the research community we also consider a state-of-the-art commercial entity resolution implementation. Our results indicate significant quality and efficiency differences between different approaches. We also find that some challenging resolution tasks such as matching product entities from online shops are not sufficiently solved with conventional approaches based on the similarity of attribute values.

Parallel Entity Resolution with Dedoop

“…In this experiment, we investigate ER strategies for the realworld dataset GS from [16] containing 64, 263 publication records from the Google Scholar search engine. The publication records are relatively unclean due to the high heterogeneity of paper citations and errors for extracting the bibliographic information from PDF files.…”

“…The publication records are relatively unclean due to the high heterogeneity of paper citations and errors for extracting the bibliographic information from PDF files. To evaluate match quality in terms of precision, recall, and F-measure, we derived a gold standard from the manually determined, perfect match result from [16] that is based on a mapping of the GS publications to corresponding DBLP publications.…”

“…Naturally, the best match quality is achieved without blocking when we evaluate the complete Cartesian product of more than two billion entity pairs. The best F-Measure is only about 80% indicating the hardness of the match problem (bibliographic ER tasks involving relatively clean data sources such as DBLP are typically solved with F-measure values of more than 90-95% [16]). The best match quality is achieved for the learning-based approach using decision trees followed by the SVM-based learning approach and, with comparable quality than SVM, the use of rule 2 .…”

“…ER techniques usually compare Lars Kolb · Erhard Rahm Institut für Informatik Universität Leipzig PF 100920 04009 Leipzig Germany E-mail: {kolb,rahm}@informatik.uni-leipzig.de pairs of entities by evaluating multiple similarity measures to make effective match decisions. As a consequence, ER is an expensive process that can take several hours or even days [16] for large datasets as it is typical for "Big Data" applications. A common approach to improve efficiency is to reduce the search space by adopting so-called blocking techniques [2].…”

Pay-as-you-go Configuration of Entity Resolution

“…Entity resolution is the task of identifying different records that represent the same entity, and is an important step in many data integration and data cleaning applications [11,21]. A single entity may come to be represented using different records for many reasons; for example, data may be integrated from independently developed sources that have overlapping collections (e.g., different retailers may have overlapping product lines), or a single organization may capture the same data repeatedly (e.g., a police force may encounter the same individual or address many times, in situations where it may be difficult to be confident of the quality of the data).…”