Overview of ETL Tools and Talend-Data Integration

Sreemathy, J; Brindha, R.; Nagalakshmi, Malathy; Suvekha, N; Ragul, N Karthick; Praveennandha, M

doi:10.1109/icaccs51430.2021.9441984

Cited by 16 publications

(15 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Companies can utilize ETL to collect data from several sources and combine it in a single, centralized place. [6]…”

Section: Etl Definitionsmentioning

confidence: 99%

“…One of the most popular ETL tools is the Informatica PowerCenter, which is available on premises and allows for connection to a number of database systems, while providing data governance control, master data management, and data masking [6]. Real-time data integration, data analytics, and B2B data fusion are also available, and are some of the benefits offered by this tool.…”

Section: Etl Tools Reviewmentioning

confidence: 99%

“…Thus, ETL abstracts the heterogeneity of data sources and provides the end user with a single access point to the storage system to submit their queries. According to Sreemathy et al [6], the entire development process is conducted to meet the objectives of enterprise performance management, application challenge development, finance, and other enterprise management sources.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Digital Transformation: After the Big Data, What Next?

Udofia¹,

Buduka²,

Akpabio

et al. 2020

Day 1 Tue, August 11, 2020

View full text Add to dashboard Cite

Our definition of ‘The Big Data’ is the ability of having all the data set at the point of decision-making. Several documented research shows that less than 5% of all the data acquired in the oil and gas industry are actually utilised. Despite these challenges most operating assets, still keep acquiring more data and creating data-load that are never utilised or put into use. This paper aims to demonstrate the reason(s) why we have this data-load and instigate a solution path in managing this challenge for value realisation. Data is a corporate asset in any organisation but the default ‘silo’ work culture in most operating asset makes it difficult to get the relevant insight from these data. Organisation must be deliberate in creating the right data architecture from the data process domain through into the distributed control system, into the data historian, then within the data platform and the visualization realm and finally into the workflow domain. It is at the workflow domain that we get the ‘game-changer’ otherwise called value. This is the differentiating factor in delivering the ‘Big Data’ into the space of benefit realisation. This was what was achieved in a field in Niger Delta and has been shared compared to global best practice. The documented result from this case study showed that any data architecture should be framed to address asset business need, right from the data process domain function through to the data historian into the workflow level where the business objective is addressed. Data must be application-centric accounting for scalability, accessibility, security and business performance. Assets need to be clear on why they are acquiring data and not just continuous acquisition of more data with the resultant effect of data-load without answering the question; what next after data acquisition? This paper provides the answer of what next after data acquisition and serves as a quick reference tool for similar challenge elsewhere, not limited to the oil and gas industry.

show abstract

“…Companies can utilize ETL to collect data from several sources and combine it in a single, centralized place. [6]…”

Section: Etl Definitionsmentioning

confidence: 99%

Section: Etl Tools Reviewmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Digital Transformation: After the Big Data, What Next?

Udofia¹,

Buduka²,

Akpabio

et al. 2020

Day 1 Tue, August 11, 2020

View full text Add to dashboard Cite

show abstract

“…[10] showcases the use of Hadoop and its ecosystem to be the most used tools for constructing data lakes, indicating their prominence in current architecture. [11] gives an overview of ETL tools and compares Talend and Informatica, both of which are data integration tools. Data quality is paramount in data pipelines as it directly impacts the reliability and usefulness of the insights derived from the data.…”

Section: Introductionmentioning

confidence: 99%

Designing Efficient Data Pipelines: A Framework for Ingestion, Processing, and Enrichment

2024

IJSREM

View full text Add to dashboard Cite

ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform) pipelines are the foundation of modern data processing, enabling efficient transformation and integration of various data sources. Despite extensive research, significant gaps remain, particularly in the areas of data processing, data quality and integrity, and security and privacy. This article provides a comprehensive review of the existing literature on prompts to identify gaps. We then present the design and implementation of an advanced combination therapy system that addresses these issues. Our pipeline combines real-time data streaming, efficient data processing, and advanced security features. The results of this study help create efficient, reliable, and secure ETL/ELT processes and provide methods that organizations can use to optimize processes and improve survey results. Keywords—BigData, Pipeline, Extract-Transform-Load(ETL),

show abstract

“…This section identified the categories and their common characteristics, which will make it easier to select the best tool for your needs. Most tools fall into various categories due to their powerful features [7].…”

Section: Introductionmentioning

confidence: 99%

Comparative Analysis of ETL Tools in Big Data Analytics

Qaiser

Farooq

Mustafa

et al. 2023

PakJET

View full text Add to dashboard Cite

Massive amounts of data are being generated these days all around the world. The majority of social networking platforms, ecommerce sites, the public and private sectors, healthcare institutions, cloud networks, and various servers are all generating immense amount of data. Collected data from various sites could be in a structured or unstructured format. Extract Transform Load (ETL) is a crucial component of the growing demand for quicker business decisions aimed at many contemporary applications. Due to the volume and speed of data, real-time ETL is built on the foundation of multi-source, unstructured data stream extraction and transformation employing disc data in dispersed environments. The entire procedure is pipelined while processing so that the final resultant data can provide some essential and useful conclusions to work on. Some analytical findings are once again helpful in making decisions. However, the produced results may differ in some numbers, graphs, and figures in many circumstances. This occurs as a result of the usage of some unrealistic tools for big data processing. This paper proposed several methods and appropriate ETL (extract, transform, and load) tools for the big data processing, which may result in appropriate analytics and conclusions from the data.

show abstract

Overview of ETL Tools and Talend-Data Integration

Cited by 16 publications

References 5 publications

Digital Transformation: After the Big Data, What Next?

Digital Transformation: After the Big Data, What Next?

Designing Efficient Data Pipelines: A Framework for Ingestion, Processing, and Enrichment

Comparative Analysis of ETL Tools in Big Data Analytics

Contact Info

Product

Resources

About