DEF - a programming language agnostic framework and execution environment for the parallel execution of library routines

Abstract

There is high demand for library routines that can be included into arbitrary programs and executed in parallel in the Cloud. So our approach is to provide a framework that supports the parallelized execution of library routines, written in different programming languages, from any platform. Our Distributed Execution Framework (DEF) allows to (1) deploy arbitrary routines into a central library and (2) integrate these library routines at runtime into user programs in a way that allows the routines to be executed in parallel in the Cloud. The programming and runtime environment of the library routine is completely transparent to the user and the chosen programming and runtime environment. DEF provides client and library APIs with primitives like search_lib(), create_task(), submit_job() which are integrated into the user's program to access the DEF runtime. DEF allows the user to configure clusters in a public/private Cloud and automatically distributes the tasks for executing the library routines on the workers of the cluster.

λHive

“…Distributed Execution Framework (DEF) (Feilhauer & Sobotka, 2016) consists of a set of applications that provide a PaaS where various clients using different programming languages can build and make use of an extensive library of highly parallelizable functions. DEF allows the construction and execution of algorithms in the cloud, making use of the processing power available there.…”