On data lake architectures and metadata management

Sawadogo, Pegdwendé; Darmont, Jérôme

doi:10.1007/s10844-020-00608-7

Cited by 140 publications

(91 citation statements)

References 35 publications

(145 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…There is a demand for stable and usually web-accessible storage [90,144,147]. Data Lakes ingest raw data in its original format from various data sources, meet their role as storage repositories, and allow users to query and explore them to extract knowledge [167]. In the Extract, Transform, Load (ETL) standard procedure, "Load" is close to "store, " and this procedure performs a critical role in data storage [162].…”

Section: Analysis Phase Key Functionsmentioning

confidence: 99%

DaLiF: a data lifecycle framework for data-driven governments

2021

View full text Add to dashboard Cite

The public sector, private firms, business community, and civil society are generating data that is high in volume, veracity, velocity and comes from a diversity of sources. This kind of data is known as big data. Public Administrations (PAs) pursue big data as “new oil” and implement data-centric policies to transform data into knowledge, to promote good governance, transparency, innovative digital services, and citizens’ engagement in public policy. From the above, the Government Big Data Ecosystem (GBDE) emerges. Managing big data throughout its lifecycle becomes a challenging task for governmental organizations. Despite the vast interest in this ecosystem, appropriate big data management is still a challenge. This study intends to fill the above-mentioned gap by proposing a data lifecycle framework for data-driven governments. Through a Systematic Literature Review, we identified and analysed 76 data lifecycles models to propose a data lifecycle framework for data-driven governments (DaliF). In this way, we contribute to the ongoing discussion around big data management, which attracts researchers’ and practitioners’ interest.

show abstract

Section: Analysis Phase Key Functionsmentioning

confidence: 99%

DaLiF: a data lifecycle framework for data-driven governments

2021

View full text Add to dashboard Cite

show abstract

“…A data lake is a large, raw data repository that stores and manages the company data bearing any format. The concept of data lake was introduced in the last decade in order to address issues related to processing big data [4]. Moreover, recently, the semantic data lakes [5] are introduced as an extension of the data lake supplying it with a semantic middleware, which allows the uniform access to original heterogeneous data sources.…”

Section: Fig 1 Modern Data Ecosystemmentioning

confidence: 99%

Arabic linked drug dataset consolidating and publishing

2021

View full text Add to dashboard Cite

The paper examines the process of creating and publishing an Arabic Linked Drug Dataset based on open drug datasets from selected Arabic countries and discusses quality issues considered in the linked data lifecycle when establishing a semantic Data Lake in the pharmaceutical domain. Through representation of the data in an open machine-readable format, the approach provides an optimum solution for information and dissemination of data and for building specialized applications. Authors contribute to opening the drug datasets from Arabic countries, interlinking the data with diverse repositories such as DrugBank, and DBpedia, and publishing it in a standard open manner that allows further integration and building different business services on top of the integrated data. This paper showcases how drug industry can take full advantage of the emerging trends for building competitive advantages. However, as is elaborated in this paper, better understanding of the specifics of the Arabic language is needed in order to extend the usage of linked data technologies in Arabic companies.

show abstract

“…Delta Lake is a storage layer for improving the reliability of data lakes [36][37][38]. Delta Lake can operate on the basis of implemented data lakes using Apache Hadoop [11], Amazon S3 [32] or Azure Data Lake Storage [39].…”

Section: Delta Lakementioning

confidence: 99%

Analysis of Big Data Storage Tools for Data Lakes based on Apache Hadoop Platform

Belov¹,

Nikulchev²

2021

IJACSA

View full text Add to dashboard Cite

When developing large data processing systems, the question of data storage arises. One of the modern tools for solving this problem is the so-called data lakes. Many implementations of data lakes use Apache Hadoop as a basic platform. Hadoop does not have a default data storage format, which leads to the task of choosing a data format when designing a data processing system. To solve this problem, it is necessary to proceed from the results of the assessment according to several criteria. In turn, experimental evaluation does not always give a complete understanding of the possibilities for working with a particular data storage format. In this case, it is necessary to study the features of the format, its internal structure, recommendations for use, etc. The article describes the features of both widely used data storage formats and the currently gaining popularity.

show abstract

On data lake architectures and metadata management

Cited by 140 publications

References 35 publications

DaLiF: a data lifecycle framework for data-driven governments

DaLiF: a data lifecycle framework for data-driven governments

Arabic linked drug dataset consolidating and publishing

Analysis of Big Data Storage Tools for Data Lakes based on Apache Hadoop Platform

Contact Info

Product

Resources

About