Reach for gold

Abstract

Duplicates in a database are one of the prime causes of poor data quality and are at the same time among the most difficult data quality problems to alleviate. To detect and remove such duplicates, many commercial and academic products and methods have been developed. The evaluation of such systems is usually in need of pre-classified results. Such gold standards are often expensive to come by (much manual classification is necessary), not representative (too small or too synthetic), and proprietary and thus preclude repetition (company-internal data). This lament has been uttered in many papers and even more paper reviews. The proposed annealing standard is a structured set of duplicate detection results, some of which are manually verified and some of which are merely validated by many classifiers. As more and more classifiers are evaluated against the annealing standard, more and more results are verified and validation becomes more and more confident. We formally define gold, silver, and the annealing standard and their maintenance. Experiments show how quickly an annealing standard converges to a gold standard. Finally, we provide an annealing standard for 750,000 CDs to the duplicate detection community.

Generating automatically labeled data for author name disambiguation: an iterative clustering method

“…2 Other than these three types, a few studies have used synthetic labeled data (e.g.,Ferreira, Gonçalves, Almeida, Laender, & Veloso, 2012;Milojević, 2013). Another noticeable labeling approach is to use the intersection set of disambiguation results by multiple algorithms(Vogel, Heise, Draisbach, Lange, & Naumann, 2014) 3 This does not imply that onlyHan et al (2004)'s data contain flaws. No other labeled data thanHan et al (2004)'s have received such intensive scrutiny for errors.…”