Metadata Management for Textual Documents in Data Lakes

Sawadogo, Pegdwendé; Kibata, Tokio; Darmont, Jérôme

doi:10.5220/0007706300720083

Cited by 22 publications

(28 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Other models exist which are not part of a specific system. However, many of these, also including that by Walker and Alrehamy, only focus on a specific topic and thus, only support a limited set of use cases which makes them non-generic, e.g., [8,11,17,20,21]. Thenceforth these models are not considered here.…”

Section: Related Work: Discussion Of Existent Metadata Modelsmentioning

confidence: 99%

See 1 more Smart Citation

HANDLE - A Generic Metadata Model for Data Lakes

Eichler

Giebler

Gröger

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

The substantial increase in generated data induced the development of new concepts such as the data lake. A data lake is a large storage repository designed to enable flexible extraction of the data's value. A key aspect of exploiting data value in data lakes is the collection and management of metadata. To store and handle the metadata, a generic metadata model is required that can reflect metadata of any potential metadata management use case, e.g., data versioning or data lineage. However, an evaluation of existent metadata models yields that none so far are sufficiently generic. In this work, we present HANDLE, a generic metadata model for data lakes, which supports the flexible integration of metadata, data lake zones, metadata on various granular levels, and any metadata categorization. With these capabilities HANDLE enables comprehensive metadata management in data lakes. We show HANDLE's feasibility through the application to an exemplary access-use-case and a prototypical implementation. A comparison with existent models yields that HANDLE can reflect the same information and provides additional capabilities needed for metadata management in data lakes.

show abstract

Section: Related Work: Discussion Of Existent Metadata Modelsmentioning

confidence: 99%

“…A central aspect of metadata management is the definition of a metadata model (e.g., [10,15,17]). By our definition a metadata model describes the relations between data and metadata elements and what metadata is collected, e.g., in the form of an explicit schema, a formal definition, or a textual description.…”

Section: Introductionmentioning

confidence: 99%

HANDLE - A Generic Metadata Model for Data Lakes

Eichler

Giebler

Gröger

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…The definition of a metadata model for data lakes also involves identifying the metadata to be considered. To this end, we extend a medatata typology that categorizes metadata into intra-object, inter-object and global metadata [23] with new types of inter-object (relationships) and global (index, event logs) metadata.…”

Section: Metadata Typologymentioning

confidence: 99%

Metadata Systems for Data Lakes: Models and Features

Sawadogo

Scholly

Favre

et al. 2019

Communications in Computer and Information Science

Self Cite

View full text Add to dashboard Cite

2 BIAL-X https://www.bial-x.com/ Abstract. Over the past decade, the data lake concept has emerged as an alternative to data warehouses for storing and analyzing big data. A data lake allows storing data without any predefined schema. Therefore, data querying and analysis depend on a metadata system that must be efficient and comprehensive. However, metadata management in data lakes remains a current issue and the criteria for evaluating its effectiveness are more or less nonexistent. In this paper, we introduce MEDAL, a generic, graph-based model for metadata management in data lakes. We also propose evaluation criteria for data lake metadata systems through a list of expected features. Eventually, we show that our approach is more comprehensive than existing metadata systems.

show abstract

“…It must provide synchronous and asynchronous communication capability. (9) The data governance layer provides a set of tools to establish and execute plans and programs for data quality control [4]. Each of the above layers is implemented with one or more frameworks of the Apache Hadoop ecosystem, e.g., Atlas 1 , HDFS 2 , HIVE 3 , OpenLdap 4 , Spark 5 , etc.…”

Section: Example Dh Projects Involving Data Lakes 21 Hyperthesaumentioning

confidence: 99%

Data lakes for digital humanities

Darmont

Favre

Loudcher

et al. 2020

Proceedings of the 2nd International Conference on Digital Tools &Amp; Uses Congress

Self Cite

View full text Add to dashboard Cite

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

show abstract

Metadata Management for Textual Documents in Data Lakes

Cited by 22 publications

References 11 publications

HANDLE - A Generic Metadata Model for Data Lakes

HANDLE - A Generic Metadata Model for Data Lakes

Metadata Systems for Data Lakes: Models and Features

Data lakes for digital humanities

Contact Info

Product

Resources

About