Analysis of Change Data Capture Method in Heterogeneous Data Sources to Support RTDW

Chandra, Harry

doi:10.1109/iccoins.2018.8510574

Cited by 9 publications

(3 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is due to hardware limitations, network protocol, limitations of transfer rate of data or whatever type of restrictions that act to increase the time to send data to the target data warehouse. [7], [18], [11], [9], [25].…”

Section: Etlmentioning

confidence: 99%

“…The immediate solution to meet real-time is to increase the data warehouse data updating frequency, that is, the ETL process can be performed more than once a day and in each application idle time [20], [7], [9]. However, this solution can have three main drawbacks: 1) data reading from operational sources: the data sources from the operational environment are used to serve a group of users that performs transactions continuously.…”

Section: Etlmentioning

confidence: 99%

“…This process is responsible for dealing with the operational data and performing the extraction, cleaning, processing, and integration of data. Thus, the data of interest available in the operational environment are sent to a data warehousing environment and stored into a multidimensional, homogeneous, and integrated database called data warehouse [7], [8], [5].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An Innovative Method to Extract Data in a Real-time Data Warehousing Environment

Vilela¹,

Ciferri²

2021

Computing Advances &Amp; Trends

View full text Add to dashboard Cite

ETL (Extract, Transform, and Load) is an essential process required to perform data extraction in knowledge discovery in databases and in data warehousing environments. The ETL process aims to gather data that is available from operational sources, process and store them into an integrated data repository. Also, the ETL process can be performed in a real-time data warehousing environment and store data into a data warehouse. This paper presents a new and innovative method named Data Extraction Magnet (DEM) to perform the extraction phase of ETL process in a real-time data warehousing environment based on non-intrusive, tag and parallelism concepts. DEM has been validated on a dairy farming domain using synthetic data. The results showed a great performance gain in comparison to the traditional trigger technique and the attendance of real-time requirements.

show abstract

Section: Etlmentioning

confidence: 99%

Section: Etlmentioning

confidence: 99%