mxkernel: A Novel System Software Stack for Data Processing on Modern Hardware

Abstract

Emerging hardware platforms are characterized by large degrees of parallelism, complex memory hierarchies, and increasing hardware heterogeneity. Their theoretical peak data processing performance can only be unleashed if the different pieces of systems software collaborate much more closely and if their traditional dependencies and interfaces are redesigned. We have developed the key concepts and a prototype implementation of a novel system software stack named mxkernel. For MxKernel, efficient large scale data processing capabilities are a primary design goal. To achieve this, heterogeneity and parallelism become first-class citizens and deep memory hierarchies are considered from the very beginning. Instead of a classical “thread” model, mxkernel provides a simpler control flow abstraction: mxtasks model closed units of work, for which mxkernel will guarantee the required execution semantics, such exclusive access to a specific object in memory. They can be a very elegant abstraction also for heterogeneity and resource sharing. Furthermore, mxtasks are annotated with metadata, such as code variants (to support heterogeneity), memory access behavior (to improve cache efficiency and support memory hierarchies), or dependencies between mxtasks (to improve scheduling and avoid synchronization cost). With precisely the required metadata available, mxkernel can provide a lightweight, yet highly efficient form of resource management, even across applications, operating system, and database. Based on the mxkernel prototype we present preliminary results from this ambitious undertaking. We argue that threads are an ill-suited control flow abstraction for our modern computer architectures and that a task-based execution model is to be favored.

Silentium! Run-Analyse-Eradicate the Noise out of the DB/OS Stack

“…Currently, there is renewed interest in building database-specific operating systems, partly motivated by such problems as unpredictability in performance. For instance, the MXKernel project [Mü20] proposes an alternative to the classic thread model, to cater to the demands of large-scale data processing. The DBOS initiative [Ca20] goes so far as to envision managing database-internal data structures inside the OS kernel.…”

Tell-Tale Tail Latencies: Pitfalls and Perils in Database Benchmarking

“…Specially crafted real-time databases [27], novel scheduling algorithms in scheduling/routing queries [24,25,45], transactional concepts [5], or query evaluation strategies [19,51], work from inside the database. Careful tailoring of the whole software stack from OS kernel to DB engine [30,33], or crafting dedicated operating systems [4,[20][21][22]36,37] to leverage the advantages of modern hardware in database system engineering (e.g., [15,29]), contribute to solutions from below the database.…”