HOUDAL: A Data Lake Implemented for Public Housing

Scholly, Etienne; Favre, Cécile; Ferey, Eric; Loudcher, Sabine

doi:10.5220/0010418200390050

Cited by 3 publications

(6 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The retrieval times for big data queries give us a conclusion that graph databases are suitable for EMEMODL. Our contribution demonstrates the importance of high flexibility and better scalability as major features, giving more extensibility and genericity, which is not supported by recent models such as [9][10][11][12][13][26][27][28]. Although the NoSQL paradigm will never completely replace the relational paradigm, it may become a better option for projects that require scalability and work with unstructured data.…”

Section: Discussionmentioning

confidence: 99%

“…When a particular concept has no equivalent, in this case, it will be marked by a dash. As illustrated in the Table 4, through the six concepts proposed by our metadata model, we will be able to generalize the different concepts suggested by other models, like [11,12,27,28]. Indeed, for instance, the concept of data entity represents the basic unit of our Furthermore, establishing extensible terminologies and determining the relationship between these different concepts is considered one of the major requirements in modeling a metadata model.…”

Section: Logical Modelmentioning

confidence: 93%

“…However, HANDLE is incomplete due to the fact that its architecture does not permit data lineage. As a result, goldMEDAL [28] is proposed to cover the gaps left by MEDAL and HANDLE. Subsequently, goldMEDAL is not considered generic in the sense that it doesn't provide data scalability.…”

Section: Metadata Modelsmentioning

confidence: 99%

“…Indeed, several concepts have been proposed in the literature to reference data sources such as data units, datasets, raw data, etc. To standardize them, we have adopted the same concept used by goldMEDAL [28] to denote raw data. It is formalized, as we can see in Eq.…”

Section: Conceptual Modelmentioning

confidence: 99%

See 3 more Smart Citations

EMEMODL: Extensible Metadata Model for Big Data Lakes

2023

IJIES

View full text Add to dashboard Cite

The evolution of the huge amount of heterogeneous data sources introduces the emergence of several fresh new concepts. Among one of the most well-known concepts emerging as a recent and trending topic in big data is the data lake. Which represents a central repository that stores heterogeneous data sources in its native format without any predefined schema. In the absence of any enforced schema, effective metadata management based on metadata models remains an active research topic to address the data lake issues knowing by a data swamp. However, the examination of existing metadata models shows that none of them proposes a complete model. In this article, we propose a generic and extensible metadata model that supports high flexibility and better scalability in the integration of metadata. With these capabilities, EMEMODL enables comprehensive metadata management for data lakes. We show EMEMODL's feasibility through a prototypical implementation based on TPC-H datasets of different sizes to prove the scaling feature, using MySQL and Neo4J. The findings of these experiments revealed encouraging results for big data queries that give us a graph database, which shows an efficient ability to manage and process large amounts of data compared to a relational database in terms of retrieval time queries and resource consumption, including cpu and memory usage.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Logical Modelmentioning

confidence: 93%

Section: Metadata Modelsmentioning

confidence: 99%

Section: Conceptual Modelmentioning

confidence: 99%

See 2 more Smart Citations

EMEMODL: Extensible Metadata Model for Big Data Lakes

2023

IJIES

View full text Add to dashboard Cite

show abstract

“…They provide descriptions and constraints on the datasets and their relationships. Metadata models can be based on various formalisms such as logic [7], graphs [13] or UML [16]. Each formalism provides a different trade-off between expressiveness and restrictiveness.…”

Section: Related Workmentioning

confidence: 99%

Preventing Technical Errors in Data Lake Analyses with Type Theory

Guyot

Leclercq

Gillet

et al. 2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

show abstract

A Temporal Metadata Management Approach for Data Lakes

Brahmia,

Grandi

et al. 2024

Information Systems Engineering and Management

View full text Add to dashboard Cite

HOUDAL: A Data Lake Implemented for Public Housing

Cited by 3 publications

References 15 publications

EMEMODL: Extensible Metadata Model for Big Data Lakes

EMEMODL: Extensible Metadata Model for Big Data Lakes

Preventing Technical Errors in Data Lake Analyses with Type Theory

A Temporal Metadata Management Approach for Data Lakes

Contact Info

Product

Resources

About