The design and implementation of OGSA-DQP: A service-based distributed query processor

Abstract

Service-based approaches are rising to prominence because of their potential to meet the requirements for distributed application development in e-business and e-science. The emergence of a service-oriented view of hardware and software resources raises the question as to how database management systems and technologies can best be deployed or adapted for use in such an environment. This paper explores one aspect of service-based computing and data management, viz., how to integrate query processing technology with a service-based architecture suitable for a Grid environment. The paper addresses this by describing in detail the design and implementation of a service-based distributed query processor. The query processor is service-based in two orthogonal senses: firstly, it supports querying over data storage and analysis resources that are made available as services, and, secondly, its internal architecture factors out as services the functionalities related to the construction and execution of distributed query plans. The resulting system both provides a declarative approach to service orchestration, and demonstrates how query processing can benefit from a service-based architecture. As well as describing and motivating the architecture used, the paper also describes usage scenarios, and, using a bioinformatics application, presents performance results that benchmark the system and illustrate the benefits provided by the service-based architecture.

Semantics and Optimization of the SPARQL 1.1 Federation Extension

“…SPARQL-DQP is a query evaluation system built on top of OGSA-DAI [3] and OGSA-DQP [10]. OGSA-DAI is a generic service-based data access, integration, transformation and delivery framework that allows executing data-centric workflows involving heterogeneous data resources.…”

“…This system, SPARQL-DQP (which stands for SPARQL Distributed Query Processing), is built on top of the OGSA-DAI and OGSA-DQP infrastructure [3,10], what provides additional robustness to deal with large amounts of data in distributed settings, supporting for example an indirect access mode that is normally used in the development of data-intensive workflows. We have evaluated our system using a small benchmark of real SPARQL 1.1 queries from the bioinformatics domain, and compared it with other similar systems, in some cases adapting the queries to their own ad-hoc SPARQL extensions, so that the benefits of our implementation can be illustrated.…”

A service-oriented system for distributed data querying and integration on Grids

“…More specifically, OGSA-DQP [5,4,6] is a service-based distributed query processor. OGSA-DQP is service-based in two orthogonal dimensions:…”

“…This fact has led to the production of vendor systems, such as Oracle11g and IBM DB2. At the level of Grid middleware infrastructure, two notable (and correlated) examples are the OGSA Data Access and Integration (OGSA-DAI) [3] and the OGSA Distributed Query Processor (OGSA-DQP) 1 [4][5][6] projects. These projects have moved toward a serviceoriented architecture quite early in their lifecycle.…”

Optimization of decentralized multi-way join queries over pipelined filtering services

“…As a consequence, there is a growing interest in systems that are capable of processing complex queries (i.e., tasks) utilizing remotely deployed services. Examples of such systems are scientific and business workflow management systems (e.g., Taverna [2], Triana [3], Pegasus [4]), service-based execution engines (e.g., OGSA-DAI/DQP [5,6], YQL 1 , BPELSE 2 , Apache ServiceMix 3 ) and data mashup platforms (e.g., Yahoo! Pipes 4 , IBM Mashup Center 5 , WSO2 Mashup Server 6 ).…”