DAMS: A Distributed Analytics Metadata Schema

Abstract

In recent years, implementations enabling Distributed Analytics (DA) have gained considerable attention due to their ability to perform complex analysis tasks on decentralised data by bringing the analysis to the data. These concepts propose privacy-enhancing alternatives to data centralisation approaches, which have restricted applicability in case of sensitive data due to ethical, legal or social aspects. Nevertheless, the immanent problem of DA-enabling architectures is the black-box-alike behaviour of the highly distributed components originating from the lack of semantically enriched descriptions, particularly the absence of basic metadata for datasets or analysis tasks. To approach the mentioned problems, we propose a metadata schema for DA infrastructures, which provides a vocabulary to enrich the involved entities with descriptive semantics. We initially perform a requirement analysis with domain experts to reveal necessary metadata items, which represents the foundation of our schema. Afterwards, we transform the obtained domain expert knowledge into user stories and derive the most significant semantic content. In the final step, we enable machine-readability via RDF(S) and SHACL serialisations. We deploy our schema in a proof-of-concept monitoring dashboard to validate its contribution to the transparency of DA architectures. Additionally, we evaluate the schema’s compliance with the FAIR principles. The evaluation shows that the schema succeeds in increasing transparency while being compliant with most of the FAIR principles. Because a common metadata model is critical for enhancing the compatibility between multiple DA infrastructures, our work lowers data access and analysis barriers. It represents an initial and infrastructure-independent foundation for the FAIRification of DA and the underlying scientific data management.

A Privacy-Preserving Distributed Analytics Platform for Health Care Data

“…This metadata provides, for example, information about the data the code is accessing or information about the Train creator. 20 …”

“…To tackle this problem of lacking information, we developed a novel metadata schema, which enriches each incorporated digital asset with detailed semantics, in one of our previous works. 20 This metadata schema—based on Resource Description Framework Schema (RDFS)—is used by our monitoring components to provide descriptive information to the actors (see orange components in Fig. 1 ).…”

“…We exemplary show the current implementations in the context of the above-mentioned PHT approach and we present novel features, which constitute valuable options to eliminate common shortcomings like nontransparency of DA ecosystems and customisability. 20 …”

Untitled

“…This metadata provides, for example, information about the data the code is accessing or information about the Train creator. 20 The Train itself is executed in a so-called Docker-in-Docker (DinD) container. With this approach, we isolate the Train execution from the host Docker engine and create sandbox-inspired runtime environment, which involves another layer of security.…”

Multi-Institutional Breast Cancer Detection Using a Secure On-Boarding Service for Distributed Analytics

“…Therefore, approaches for Distibuted Analytics (DA) have come into focus [7,[9][10][11][12][13][14][15]. The advantages of these approaches stem from a paradigm shift in current data analyses.…”