Evaluating end-to-end optimization for data analytics applications in weld

Abstract

Modern analytics applications use a diverse mix of libraries and functions. Unfortunately, there is no optimization across these libraries, resulting in performance penalties as high as an order of magnitude in many applications. To address this problem, we proposed Weld, a common runtime for existing data analytics libraries that performs key physical optimizations such as pipelining under existing, imperative library APIs. In this work, we further develop the Weld vision by designing an automatic adaptive optimizer for Weld applications, and evaluating its impact on realistic data science workloads. Our optimizer eliminates multiple forms of overhead that arise when composing imperative libraries like Pandas and NumPy, and uses lightweight measurements to make data-dependent decisions at runtime in ad-hoc workloads where no statistics are available, with sub-second overhead. We also evaluate which optimizations have the largest impact in practice and whether Weld can be integrated into libraries incrementally. Our results are promising: using our optimizer, Weld accelerates data science workloads by up to 23× on one thread and 80× on eight threads, and its adaptive optimizations provide up to a 3.75× speedup over rule-based optimization. Moreover, Weld provides benefits if even just 4-5 operators in a library are ported to use it. Our results show that common runtime designs like Weld may be a viable approach to accelerate analytics.

An intermediate representation for optimizing machine learning pipelines

“…Python and R offer popular libraries that are easy to use and provide fast development cycles. These libraries are embedded shallowly [30] in the host language, i.e., they are executed as-is, without any inter-library optimizations and support for large data [53]. General purpose dataflow systems [71,2] provide second-order functions (e.g., map and reduce) to transform collections via userdefined functions (UDFs).…”

Optimizing data-intensive computations in existing libraries with split annotations

“…We discuss workloads below. NumPy Numerical Analysis (Figures 4a-d [55]. The number in parentheses shows the number of library API calls.…”

“…We evaluate the end-to-end performance benefits of SAs using Mozart on a suite of 15 data analytics workloads (of which four are repeated in NumPy and Intel MKL). Eight of the benchmarks are taken from the Weld evaluation [55], which obtained them from popular GitHub repositories, Kaggle competitions, and online tutorials. We also evaluate an additional numerical analysis workload (Shallow Water) over matrix operations, taken from the Bohrium paper [46] (Bohrium is an optimizing NumPy compiler that we compare against here).…”

“…Second, in order to pipeline data across functions, the runtime requires a lazily evaluated dataflow graph of the annotated functions in an application. While existing systems [17,32,46,55] require library developers to change their functions to explicitly construct and execute such a graph using an API, our goal is to enable these optimizations without library modifications. This is challenging since most applications will not only make calls to annotated library functions, but also execute arbitrary un-annotated code that cannot be pipelined.…”

Arc: an IR for batch and stream programming

“…Most models for data programming typically involve different types of side-effect free transformations on data collections and are adequately covered by prior work on Weld [12,13] among others. The purpose of Arc is to complement Weld with data stream semantics.…”