Compliant geo-distributed data processing in action

Abstract

In this paper we present our work on compliant geo-distributed data processing. Our work focuses on the new dimension of dataflow constraints that regulate the movement of data across geographical or institutional borders. For example, European directives may regulate transferring only certain information fields (such as non personal information) or aggregated data. Thus, it is crucial for distributed data processing frameworks to consider compliance with respect to dataflow constraints derived from these regulations. We have developed a compliance-based data processing framework, which (i) allows for the declarative specification of dataflow constraints, (ii) determines if a query can be translated into a compliant distributed query execution plan, and (iii) executes the compliant plan over distributed SQL databases. We demonstrate our framework using a geo-distributed adaptation of the TPC-H benchmark data. Our framework provides an interactive dashboard, which allows users to specify dataflow constraints, and analyze and execute compliant distributed query execution plans.

Delay-Resistant Geo-Distributed Analytics

“…SDTP [34] considers the dynamicity of the network bandwidth and computation parallelism to optimize job response time. However, the work of [35] showed that it is possible to put regulation on the data movement in geodistributed datacenters. All the above works assume that the size and location of data are available before the query execution.…”