Hare

Abstract

Hare is a new file system that provides a POSIX-like interface on multicore processors without cache coherence. Hare allows applications on different cores to share files, directories, and file descriptors. The challenge in designing Hare is to support the shared abstractions faithfully enough to run applications that run on traditional shared-memory operating systems, with few modifications, and to do so while scaling with an increasing number of cores.To achieve this goal, Hare must support features (such as shared file descriptors) that traditional network file systems don't support, as well as implement them in a way that scales (e.g., shard a directory across servers to allow concurrent operations in that directory). Hare achieves this goal through a combination of new protocols (including a 3-phase commit protocol to implement directory operations correctly and scalably) and leveraging properties of non-cache-coherent multiprocessors (e.g., atomic low-latency message delivery and shared DRAM).An evaluation on a 40-core machine demonstrates that Hare can run many challenging Linux applications (including a mail server and a Linux kernel build) with minimal or no modifications. The results also show these applications achieve good scalability on Hare, and that Hare's techniques are important to achieving scalability.

Sharing non‐cache‐coherent memory with bounded incoherence

“…Argo, 40 a software distributed shared memory system, distributes coherence decisions using self‐invalidation and self‐downgrade combined with hierarchical queue delegation locks. Hare 10 uses message passing to implement a distributed file‐system across nodes in a non‐CC system. libMPNode 43 implements an OpenMP runtime for incoherent domains.…”

“…Figure 1 depicts a taxonomy of trade‐offs between the hardware support for consistency via cache coherency and the scalability of distributed systems. Most prior research focuses on the right end of the spectrum, 9‐11 leaving alternatives in the design space unexplored. As pointed out by Harris, 12 solely relying on message passing does not suit some workloads, and what programming models are appropriate for combining message passing with shared memory is still an open research question.…”

libMPNode

“…Because of these trends and due to the complexities of scaling interconnects to larger core counts [17], the systems community has experienced new interest in moving from software architectures for fully cache-coherent systems to those composed of multiple incoherent domains, where cache coherence is provided only within a domain, as shown in Figure 1. Recent works have developed software architectures for mobile SoCs containing incoherent CPU cores [1,19], servers with incoherent compute elements [3,4,12] or even rack-scale clusters [21][22][23]. In these architectures, the software treats each cache coherence domain as a node and provides cross-node execution and memory coherence transparently to applications.…”