Integrating distributed data sources with OGSA–DAI DQP and V
            <scp>iews</scp>

Dobrzelecki, Bartosz; Krause, Amrey; Hume, Alastair; Grant, Alistair; Antonioletti, Mario; Alemu, Tilaye Y.; Atkinson, Malcolm; Jackson, Michael; Theocharopoulos, Elias

doi:10.1098/rsta.2010.0166

Cited by 21 publications

(20 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some of the existing engines supporting the SPARQL 1.1 Federated Query extension are ARQ 12 , RDF-Query 13 , Rasqal RDF query Library 14 and ANAP-SID [10] among others. There are also other systems which implement a distributed query processing system for SPARQL like DARQ [4], Networked Graphs [5], SPLENDID [37], FedX 1.1 [6], the system by Ladwig et al [38] and SemWIQ [39], but they do not follow the official SPARQL 1.1 Federation specification.…”

Section: Other Federated Sparql Querying Processing Systemsmentioning

confidence: 99%

“…(which stands for SPARQL Distributed Query Processing), is built on top of the OGSA-DAI and OGSA-DQP infrastructures [12,13] that allow dealing with large amounts of data in distributed settings, supporting for example an indirect access mode that is normally used in the development of data-intensive workflows. In summary, the main contributions of this paper are:…”

mentioning

confidence: 99%

See 1 more Smart Citation

Federating queries in SPARQL 1.1: Syntax, semantics and evaluation

Buil-Aranda

Arenas²,

Corcho

et al. 2013

Journal of Web Semantics

View full text Add to dashboard Cite

Given the sustained growth that we are experiencing in the number of SPARQL endpoints available, the need to be able to send federated SPARQL queries across these has also grown. To address this use case, the W3C SPARQL working group is defining a federation extension for SPARQL 1.1 which allows for combining graph patterns that can be evaluated over several endpoints within a single query. In this paper, we describe the syntax of that extension and formalize its semantics. Additionally, we describe how a query evaluation system can be implemented for that federation extension, describing some static optimization techniques and reusing a query engine used for data-intensive science, so as to deal with large amounts of intermediate and final results. Finally we carry out a series of experiments that show that our optimizations speed up the federated query evaluation process. ing complex queries by combining data from different, sometimes heterogeneous and often physically distributed datasets. SPARQL endpoints are RESTful services that accept queries over HTTP written in the SPARQL query language [1,2] adhering to the SPARQL protocol [3], as defined by the respective W3C recommendation documents. However, the current SPARQL recommendation has an important limitation in terms of defining and executing queries that span across distributed datasets, since it hides the physical distribution of data across endpoints, and has normally been used for querying isolated endpoints. Hence users willing to federate queries across a number of SPARQL endpoints have been forced to create ad-hoc extensions of the query language and protocol, to include additional information about data sources in the configuration of their SPARQL endpoint servers [4,5,6] or to devise engineering solutions where data from remote endpoints is copied into the endpoint being queried. Given the need to address these types of queries, the SPARQL working group has proposed a query federation extension for the upcoming SPARQL 1.1 language [7] which is now under discussion in order to generate a new W3C recommendation in the coming months.

show abstract

Section: Other Federated Sparql Querying Processing Systemsmentioning

confidence: 99%

mentioning

confidence: 99%

Federating queries in SPARQL 1.1: Syntax, semantics and evaluation

Buil-Aranda

Arenas²,

Corcho

et al. 2013

Journal of Web Semantics

View full text Add to dashboard Cite

show abstract

“…The prototype implementation of the ADMIRE framework currently supports enactment over two platforms: Open Grid Services Architecture-Database Access and Integration (OGSA-DAI) [13] and Unified System Management Technology (USMT) from Fujitsu Laboratories of Europe [14]. OGSA-DAI is a platform solution for distributed data access and management.…”

Section: (B) Prototype Implementationmentioning

confidence: 99%

Validation and mismatch repair of workflows through typed data streams

Yaikhom

Atkinson

Hemert

et al. 2011

Phil. Trans. R. Soc. A.

View full text Add to dashboard Cite

The type system of a language guarantees that all of the operations on a set of data comply with the rules and conditions set by the language. While language typing is a fundamental requirement for any programming language, the typing of data that flow between processing elements within a workflow is currently being treated as optional. In this paper, we introduce a three-level type system for typing workflow data streams. These types are parts of the Data Intensive System Process Engineering Language programming language, which empowers users with the ability to validate the connections inside a workflow composition, and apply appropriate data type conversions when necessary. Furthermore, this system enables the enactment engine in carrying out type-directed workflow optimizations.

show abstract

“…An experiment with EURExpress-II [22] using OGSA-DAI [14] described in Section 5. OGSA-DAI is an extensible framework which supports data streaming of heterogeneous data from multiple sources, such as: relational, XML and RDF databases, and file systems.…”

Section: Introductionmentioning

confidence: 99%

Towards optimising distributed data streaming graphs using parallel streams

Liew

Atkinson

Hemert

et al. 2010

Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing

View full text Add to dashboard Cite

Modern scientific collaborations have opened up the opportunity of solving complex problems that involve multidisciplinary expertise and large-scale computational experiments. These experiments usually involve large amounts of data that are located in distributed data repositories running various software systems, and managed by different organisations. A common strategy to make the experiments more manageable is executing the processing steps as a workflow. In this paper, we look into the implementation of fine-grained data-flow between computational elements in a scientific workflow as streams. We model the distributed computation as a directed acyclic graph where the nodes represent the processing elements that incrementally implement specific subtasks. The processing elements are connected in a pipelined streaming manner, which allows task executions to overlap. We further optimise the execution by splitting pipelines across processes and by introducing extra parallel streams. We identify performance metrics and design a measurement tool to evaluate each enactment. We conducted experiments to evaluate our optimisation strategies with a real world problem in the Life Sciences-EURExpress-II. The paper presents our distributed data-handling model, the optimisation and instrumentation strategies and the evaluation experiments. We demonstrate linear speed up and argue that this use of data-streaming to enable both overlapped pipeline and parallelised enactment is a generally applicable optimisation strategy.

show abstract

Integrating distributed data sources with OGSA–DAI DQP and V iews

Cited by 21 publications

References 3 publications

Federating queries in SPARQL 1.1: Syntax, semantics and evaluation

Federating queries in SPARQL 1.1: Syntax, semantics and evaluation

Validation and mismatch repair of workflows through typed data streams

Towards optimising distributed data streaming graphs using parallel streams

Contact Info

Product

Resources

About