Efficient Compilation and Execution of JVM-Based Data Processing Frameworks on Heterogeneous Co-Processors

Kotselidis, Christos; Diamantopoulos, Sotiris; Akrivopoulos, Orestis; Rosenfeld, Viktor; Doka, Katerina; Mohammed, Hazeef; Mylonas, Georgios; Spitadakis, Vassilis; Morgan, W. A.

doi:10.23919/date48585.2020.9116246

Cited by 2 publications

(2 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Vispark [11] is a Python-like language that translates the source code to execute on GPUs. Finally, Kotselidis et al [37] presents a system architecture that employs TornadoVM as a means to offer transparent compilation and acceleration of Flink applications on heterogeneous devices. However, that work does not focus on the challenges regarding the transparent integration of hardware acceleration in Flink and it presents a preliminary performance evaluation for k-means on a single node with a GPU.…”

Section: Related Workmentioning

confidence: 99%

Enabling Transparent Acceleration of Big Data Frameworks Using Heterogeneous Hardware

et al. 2022

View full text Add to dashboard Cite

The ever-increasing demand for high performance Big Data analytics and data processing, has paved the way for heterogeneous hardware accelerators, such as Graphics Processing Units (GPUs) and Field Programmable Gate Arrays (FPGAs), to be integrated into modern Big Data platforms. Currently, this integration comes at the cost of programmability since the end-user Application Programming Interface (APIs) must be altered to access the underlying heterogeneous hardware. For example, current Big Data frameworks, such as Apache Spark, provide a new API that combines the existing Spark programming model with GPUs. For other Big Data frameworks, such as Flink, the integration of GPUs and FPGAs is achieved via external API calls that bypass their execution models completely. In this paper, we rethink current Big Data frameworks from a systems and programming language perspective, and introduce a novel co-designed approach for integrating hardware acceleration into their execution models. The novelty of our approach is attributed to two key design decisions: a) support for arbitrary User Defined Functions (UDFs), and b) no modifications to the user level API. The proposed approach has been prototyped in the context of Apache Flink, and enables unmodified applications written in Java to run on heterogeneous hardware, such as GPU and FPGAs, transparently to the users. The performance evaluation of the proposed solution has shown performance speedups of up to 65x on GPUs and 184x on FPGAs for suitable workloads of standard benchmarks and industrial use cases against vanilla Flink running on traditional multi-core CPUs.

show abstract

Section: Related Workmentioning

confidence: 99%

Enabling Transparent Acceleration of Big Data Frameworks Using Heterogeneous Hardware

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Ideally, programmers want to make use of as many resources as possible, primarily to increase performance and save energy [85]. However, contemporary systems do not provide a homogeneous programming environment that at the same time abstracts the heterogeneous hardware characteristics and yet allows taking advantage of all heterogeneous resources available within a computer system [84,86].…”

Section: Deploying In Heterogeneous Computing Environmentsmentioning

confidence: 99%

A Smart Water Metering Deployment Based on the Fog Computing Paradigm

Amaxilatis¹,

Chatzigiannakis

Tselios

et al. 2020

Applied Sciences

View full text Add to dashboard Cite

In this paper, we look into smart water metering infrastructures that enable continuous, on-demand and bidirectional data exchange between metering devices, water flow equipment, utilities and end-users. We focus on the design, development and deployment of such infrastructures as part of larger, smart city, infrastructures. Until now, such critical smart city infrastructures have been developed following a cloud-centric paradigm where all the data are collected and processed centrally using cloud services to create real business value. Cloud-centric approaches need to address several performance issues at all levels of the network, as massive metering datasets are transferred to distant machine clouds while respecting issues like security and data privacy. Our solution uses the fog computing paradigm to provide a system where the computational resources already available throughout the network infrastructure are utilized to facilitate greatly the analysis of fine-grained water consumption data collected by the smart meters, thus significantly reducing the overall load to network and cloud resources. Details of the system’s design are presented along with a pilot deployment in a real-world environment. The performance of the system is evaluated in terms of network utilization and computational performance. Our findings indicate that the fog computing paradigm can be applied to a smart grid deployment to reduce effectively the data volume exchanged between the different layers of the architecture and provide better overall computational, security and privacy capabilities to the system.

show abstract

Efficient Compilation and Execution of JVM-Based Data Processing Frameworks on Heterogeneous Co-Processors

Cited by 2 publications

References 14 publications

Enabling Transparent Acceleration of Big Data Frameworks Using Heterogeneous Hardware

Enabling Transparent Acceleration of Big Data Frameworks Using Heterogeneous Hardware

A Smart Water Metering Deployment Based on the Fog Computing Paradigm

Contact Info

Product

Resources

About