Development of a real-time data quality monitoring system using embedded intelligence

Bethem, T.; Evans, Marlene B.; Vafaie, H.; Shaughnessy, Michael

doi:10.1109/oceans.2002.1191909

Cited by 1 publication

(2 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In detail, being able to track the data cleansing process in real-time will have these advantages (Cardoso, 2004;Bethem et al, 2002):…”

Section: Introductionmentioning

confidence: 99%

“…In Bethem et al (2002), the authors proposed a real-time data quality monitoring system for NOAA Ocean Service's oceanographic operational products based on case-based and rule-based reasoning. Another commercial data quality monitoring is offered by Trillium Software (2011) which delivers the ability to rate and score the quality of the data to help assess the accuracy of the data.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Real-Time Data Quality Monitoring System for Data Cleansing

Varol

Neumann

2012

International Journal of Business Intelligence Research

View full text Add to dashboard Cite

To assist business intelligence companies dealing with data preparation problems, different approaches have been developed to handle the dirty data. However, these data cleansing approaches do not have real-time monitoring capabilities. Therefore, business intelligence companies and their clients are not able to predict the final outcome before running all business process. This yields an extra cost for the company if the data are highly corrupted. Therefore, to reduce cost for these types of businesses, the authors design a framework that monitors the quality attributes during the data cleansing process. Moreover, the system provides feedback to the user and allows the user to restructure the workflow based on quality attributes. The main concept of the framework is based on client-server architecture that uses multithreading to allow real-time monitoring of the process. A child thread is dedicated to run and another is dedicated to monitor the processes and give feedback to the user. The real-time monitoring system not only displays the cleansing process done on the data set, but also estimates the risk propagation probabilities in the data cleansing process. De-duplication elimination, address normalization, spelling correction for personal names, and non-ASCII character removal techniques are employed.

show abstract

“…In detail, being able to track the data cleansing process in real-time will have these advantages (Cardoso, 2004;Bethem et al, 2002):…”

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%