Evaluating a model for cost-effective data quality management in a real-world CRM setting

Challenges for Value-driven Semantic Data Quality Management

“…more expressive schema (Mendel-Gleason, 2015), that the intrinsic quality metrics that the majority of such tools focus on (Zaveri, 2015) are not the locus of business or organisational value in the datasets (Evan, 2005). Even the best data quality processes and tools require human oversight to be most effective (Brous, 2016) and as the number and variety of datasets increases (especially in a world dominated by Big Data) it is not scaleable to try and improve data quality uniformly (Evan, 2010). Instead some means must be developed to focus the attention of automated tools on the places where they can do the most good with the least investment of effort.…”

“…Even with some progress on tools extrinsic measures like availability of licensing information (Neumaier, 2016), the real gains in application of data quality tools will be at the interface between addressing business needs (Schork, 2009), supporting domain experts rather than information architects (Mosley, 2010) and methods to focus on the available tools and people on the most relevant data quality issues rather than wasting effort on uniform metric improvements that might not even feed into business goals (Evan, 2010). There is a direct parallel between this situation and the author's track record on bridging the gaps in human involvement in semantic mapping processes (Conroy, 2009, Debruyne, 2013 in contrast to the majority of the research in ontology matching which focuses on improving low-level matching algorithms (Shvaiko, 2013).…”

“…Previous work on metadata and business value as drivers for data quality has focused on pre-semantic technology for data warehousing (Helfert, 2002, Shankaranarayan, 2003, organisational decision support systems (Evan, 2005, Evan, 2010, Schork, 2009, Tallon, 2007, Viscusi, 2014 as opposed to our challenges for tool automation. There are however related active research topics like file-retention strategies based on file metadata (Wijnhoven, 2014) and autonomic data lifecycle management (Chen, 2005).…”

“…By building on Chen's work on value-based autonomic data lifecycle management (Chen, 2005) and Even et al's approach to value-based data quality management for organisational decision support (Evan, 2010), it may be possible to support automated selection, prioritisation and orchestration of data quality tools.…”

“…Moody previously observed that "100% accurate information is rarely required in a business context" (Moody, 1999) so it is impractical (and unprofitable) to blindly try to achieve "high quality data" across the board (Evan, 2010). Investment in data quality can be seen as insurance against the risk that your data is not "fit for use".…”