Conceptual modeling for ETL processes

Vassiliadis, Panos; Simitsis, Alkis; Skiadopoulos, Spiros

doi:10.1145/583890.583893

Cited by 306 publications

(188 citation statements)

References 22 publications

Supporting

Mentioning

149

Contrasting

Unclassified

Order By: Relevance

“…The complexity of extraction transformation and load lies in the mapping of data from source to destination. As indicated by Alkis Simitsis [1] distributed. In the middle we have Data Staging Area (DSA) which encapsulates the major transformation and the last is the Data Warehouse representation.…”

Section: Introductionmentioning

confidence: 94%

“…As a first attempt author [16] had separated warehouse conceptual schema and ETL conceptual schema. Author in [1] has given notation but has not indicated representations for PPDM. In this paper we propose a method that introduce additional notation in design which can be used to represent PPDM representations.…”

Section: Index Terms-etl Data Warehouse Ppdm Conceptual Modelingmentioning

confidence: 99%

“…Extraction deals with the retrieval of data from various sources. These sources of data can be of different format and different location in space [1]. Each of these locations must be analyzed before it is loaded on to the data warehouse.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An Extended Conceptual Modeling for ETL Processes in Privacy Preserving Data Mining

P¹,

Kumar²,

Kavya³

2012

IJFCC

View full text Add to dashboard Cite

Abstract-Extraction transformation load is an important stage of a data warehouse process. It involves retrieval of information from multiple sources, applying transformation to map with the schema of the data warehouse. Privacy preserving data mining has become a prerequisite for all the present data warehouse projects. In this paper we provide a foundation on embedding privacy to conceptual modeling by indicating additional notations which ensures faster design.

show abstract

Section: Introductionmentioning

confidence: 94%

Section: Index Terms-etl Data Warehouse Ppdm Conceptual Modelingmentioning

confidence: 99%

See 1 more Smart Citation

An Extended Conceptual Modeling for ETL Processes in Privacy Preserving Data Mining

P¹,

Kumar²,

Kavya³

2012

IJFCC

View full text Add to dashboard Cite

show abstract

“…Vassiliadis et al [19] investigate generic properties present in ETL activities across different ETL implementations and, based on these properties, construct a taxonomy of ETL concepts. Vassiliadis et al and Skoutas & Simitsis [19,15] use a categorisation of operations from different systems to capture the relationship between input and output definitions.…”

Section: Etl Conceptual Modelsmentioning

confidence: 99%

Representing Interoperable Provenance Descriptions for ETL Workflows

Freitas

Kämpgen

Oliveira

et al. 2015

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. The increasing availability of data on the Web provided by the emergence of Web 2.0 applications and, more recently by Linked Data, brought additional complexity to data management tasks, where the number of available data sources and their associated heterogeneity drastically increases. In this scenario, where data is reused and repurposed on a new scale, the pattern expressed as Extract-Transform-Load (ETL) emerges as a fundamental and recurrent process for both producers and consumers of data on the Web. In addition to ETL, provenance, the representation of source artifacts, processes and agents behind data, becomes another cornerstone element for Web data management, playing a fundamental role in data quality assessment, data semantics and facilitating the reproducibility of data transformation processes. This paper proposes the convergence of these two Web data management concerns, introducing a principled provenance model for ETL processes in the form of a vocabulary based on the Open Provenance Model (OPM) standard and focusing on the provision of an interoperable provenance model for ETL environments. The proposed ETL provenance model is instantiated in a real-world sustainability reporting scenario.

show abstract

“…Any hand coded ETL module can not be used on real life data management until and unless its performance has been assured by testing it rigorously [14]. To identify prime testing zones for a hand coded ETL tool the authors coded an ETL tool which is capable of extracting data from databases like Oracle, SQL, MS Access, MS Excel and flat files like those of MS word [12] [13]. This tool is capable of making predefined transformations and data purification to the extent possible for managing the database.…”

mentioning

confidence: 99%

Statistically Analyzing the Impact of Automated ETL Testing on the Data Quality of a Data Warehouse

Singh¹,

Singh²

2009

IJCEE

View full text Add to dashboard Cite

Abstract--For truthful reporting and decision-making a major challenge in data warehouse industry is to ensure quality data. The Extraction, Transformation and Loading (ETL) module is crucial to attain high quality data for a data warehouse. In-house development of ETL solutions with improvised algorithms may result in unknown errors at logical or technical levels. To assure data quality one has to understand the prevailing data quality assurance practices. This paper is intended to empirically analyze the impact of automated ETL testing on the data quality of the data warehouse. The data quality was observed before and after the induction of automated ETL testing. Statistical analysis indicated a substantial escalation in data quality after the induction of automated ETL testing.

show abstract

Conceptual modeling for ETL processes

Cited by 306 publications

References 22 publications

An Extended Conceptual Modeling for ETL Processes in Privacy Preserving Data Mining

An Extended Conceptual Modeling for ETL Processes in Privacy Preserving Data Mining

Representing Interoperable Provenance Descriptions for ETL Workflows

Statistically Analyzing the Impact of Automated ETL Testing on the Data Quality of a Data Warehouse

Contact Info

Product

Resources

About