Multi-engine Analytics with IReS

Doka, Katerina; Mytilinis, Ioannis; Papailiou, Nikolaos; Giannakouris, Victor; Tsoumakos, Dimitrios; Koziris, Nectarios

doi:10.1007/978-3-030-24124-7_9

Cited by 1 publication

(1 citation statement)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, [7] requires expertise from users for deciding when to use a data processing platform and the design of [9] is not flexible enough to allow for continuous extensions new platforms, i.e., developers have to modify the source code. IReS [6] on the other hand, provides a flexible and automatic way to choose data processing platforms. However, in contrast to Wayang, it focuses more on coarsegrained operators (e.g., k-means instead of filter, map) and provides 1-to-1 mappings from abstract to execution operators, which may lead to suboptimal execution plans.…”

Section: Related Workmentioning

confidence: 99%

Apache Wayang: A Unified Data Analytics Framework

Beedkar,

Contreras-Rojas,

Gavriilidis

et al. 2023

SIGMOD Rec.

View full text Add to dashboard Cite

The large variety of specialized data processing platforms and the increased complexity of data analytics has led to the need for unifying data analytics within a single framework. Such a framework should free users from the burden of (i) choosing the right platform( s) and (ii) gluing code between the different parts of their pipelines. Apache Wayang (Incubating) is the only open-source framework that provides a systematic solution to unified data analytics by integrating multiple heterogeneous data processing platforms. It achieves that by decoupling applications from the underlying platforms and providing an optimizer so that users do not have to specify the platforms on which their pipeline should run. Wayang provides a unified view and processing model, effectively integrating the hodgepodge of heterogeneous platforms into a single framework with increased usability without sacrificing performance and total cost of ownership. In this paper, we present the architecture ofWayang, describe its main components, and give an outlook on future directions.

show abstract

Section: Related Workmentioning

confidence: 99%