Distributed Semantic Analytics Using the SANSA Stack

Lehmann, Jens; Sejdiu, Gëzim; Bühmann, Lorenz; Westphal, Patrick; Stadler, Claus; Ermilov, Ivan; Bin, Simon; Chakraborty, Nilesh; Saleem, Muhammad; Ngomo, Axel-Cyrille Ngonga; Jabeen, Hajira

doi:10.1007/978-3-319-68204-4_15

Cited by 65 publications

(55 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The work done here (available under Apache License 2.0) has been integrated into SANSA [16], an open source 12 data flow processing engine for scalable processing of large-scale RDF datasets. SANSA uses Spark offering fault-tolerant, highly available and scalable approaches to process massive sized datasets efficiently.…”

Section: Methodsmentioning

confidence: 99%

A Scalable Framework for Quality Assessment of RDF Datasets

Sejdiu

Rula

Lehmann

et al. 2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Over the last years, Linked Data has grown continuously. Today, we count more than 10,000 datasets being available online following Linked Data standards. These standards allow data to be machine readable and inter-operable. Nevertheless, many applications, such as data integration, search, and interlinking, cannot take full advantage of Linked Data if it is of low quality. There exist a few approaches for the quality assessment of Linked Data, but their performance degrades with the increase in data size and quickly grows beyond the capabilities of a single machine. In this paper, we present DistQualityAssessment -an open source implementation of quality assessment of large RDF datasets that can scale out to a cluster of machines. This is the first distributed, in-memory approach for computing different quality metrics for large RDF datasets using Apache Spark. We also provide a quality assessment pattern that can be used to generate new scalable metrics that can be applied to big data. The work presented here is integrated with the SANSA framework and has been applied to at least three use cases beyond the SANSA community. The results show that our approach is more generic, efficient, and scalable as compared to previously proposed approaches.

show abstract

Section: Methodsmentioning

confidence: 99%

A Scalable Framework for Quality Assessment of RDF Datasets

Sejdiu

Rula

Lehmann

et al. 2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…SPIRIT [7], SPECIAL [8], and the work presented in [19] have concepts related to data subjects, purposes, and data processing activities that are relevant. Those vocabularies were, however, mostly developed for a posteriori compliance analysis of logs or assessing operations before their execution.…”

Section: Consent Ontology: Related and Prior Workmentioning

confidence: 99%

“Just-in-time” generation of datasets by considering structured representations of given consent for GDPR compliance

et al. 2020

View full text Add to dashboard Cite

Data processing is increasingly becoming the subject of various policies and regulations, such as the European General Data Protection Regulation (GDPR) that came into effect in May 2018. One important aspect of GDPR is informed consent, which captures one's permission for using one's personal information for specific data processing purposes. Organizations must demonstrate that they comply with these policies. The fines that come with non-compliance are of such importance that it has driven research in facilitating compliance verification. The state-of-the-art primarily focuses on, for instance, the analysis of prescriptive models and posthoc analysis on logs to check whether data processing is compliant to GDPR. We argue that GDPR compliance can be facilitated by ensuring datasets used in processing activities are compliant with consent from the very start. The problem addressed in this paper is how we can generate datasets that comply with given consent "just-in-time". We propose RDF and OWL ontologies to represent the consent that an organization has collected and its relationship with data processing purposes. We use this ontology to annotate schemas, allowing us to generate declarative mappings that transform (relational) data into RDF driven by the annotations. We furthermore demonstrate how we can create compliant datasets by altering the results of the mapping. The use of RDF and OWL allows us to implement the entire process in a declarative manner using SPARQL. We have integrated all components in a service that furthermore captures provenance information for each step, further contributing to the transparency that is needed towards facilitating compliance verification. We demonstrate the approach with a synthetic dataset simulating users (re-)giving, withdrawing, and rejecting their consent on data processing purposes of systems. In summary, it is argued that B Christophe Debruyne debruync@tcd.ie Harshvardhan J. Pandit 123 3616 C. Debruyne et al.the approach facilitates transparency and compliance verification from the start, reducing the need for posthoc compliance analysis common in the state-of-the-art.

show abstract

“…The vocabularies in these studies and the work presented in [15] have classes for representing data processing activities (provenance) as well as a more fine-grained representation for the data that people wanted to share. In this study, it sufficed to model the relationship between consent, purpose and dataset.…”

Section: Related Workmentioning

confidence: 99%

Towards Generating Policy-Compliant Datasets

Debruyne

Pandit

Lewis

et al. 2019

2019 IEEE 13th International Conference on Semantic Computing (ICSC)

View full text Add to dashboard Cite

The development of intelligent (e.g., AI-based) applications increasingly requires governance models and processes, as financial legal sanctions are more and more being associated with violation of policies. We propose an ontology representing the informed consent that was collected by an organization and argue how it can be used to assess a dataset prior its use in any type of data processing activities. We demonstrate the utility of our ontology using a particular scenario, where datasets are generated "just in time" for a particular purpose such as sending newsletters. This scenario shows how data processing activities can be managed to in such a way as to support compliance verification. This paper furthermore compares the contributions to related work and positions it into prior work concerned with the broader problem of prescribing and analyzing compliance.

show abstract

Distributed Semantic Analytics Using the SANSA Stack

Cited by 65 publications

References 18 publications

A Scalable Framework for Quality Assessment of RDF Datasets

A Scalable Framework for Quality Assessment of RDF Datasets

“Just-in-time” generation of datasets by considering structured representations of given consent for GDPR compliance

Towards Generating Policy-Compliant Datasets

Contact Info

Product

Resources

About