ThymesisFlow: A Software-Defined, HW/SW co-Designed Interconnect Stack for Rack-Scale Memory Disaggregation

Pinto, Christian; Syrivelis, Dimitris; Gazzetti, Michele; Koutsovasilis, Panos; Reale, Andrea; Katrinis, Kostas; Hofstee, H. Peter

doi:10.1109/micro50266.2020.00075

Cited by 44 publications

(33 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A similar technology is also being developed by IBM. Their OpenCAPI [10] cache-coherent CPU-FPGA interface is already used in recent research work on disaggregated memory [61]. Second, there is an ongoing collaborative effort from multiple hardware vendors to derive a specification for a new peripheral interface with cache coherency support (CXL) for future devices.…”

Section: Discussionmentioning

confidence: 99%

Dagger: Towards Efficient RPCs in Cloud Microservices With Near-Memory Reconfigurable NICs

Lazarev

Adit

Xiang

et al. 2020

IEEE Comput. Arch. Lett.

View full text Add to dashboard Cite

The ongoing shift of cloud services from monolithic designs to microservices creates high demand for efficient and high performance datacenter networking stacks, optimized for fine-grained workloads. Commodity networking systems based on software stacks and peripheral NICs introduce high overheads when it comes to delivering small messages.We present Dagger, a hardware acceleration fabric for cloud RPCs based on FPGAs, where the accelerator is closely-coupled with the host processor over a configurable memory interconnect. The three key design principle of Dagger are: (1) offloading the entire RPC stack to an FPGA-based NIC, (2) leveraging memory interconnects instead of PCIe buses as the interface with the host CPU, and (3) making the acceleration fabric reconfigurable, so it can accommodate the diverse needs of microservices. We show that the combination of these principles significantly improves the efficiency and performance of cloud RPC systems while preserving their generality. Dagger achieves 1.3 − 3.8× higher per-core RPC throughput compared to both highly-optimized software stacks, and systems using specialized RDMA adapters. It also scales up to 84 Mrps with 8 threads on 4 CPU cores, while maintaining state-ofthe-art s-scale tail latency. We also demonstrate that large thirdparty applications, like memcached and MICA KVS, can be easily ported on Dagger with minimal changes to their codebase, bringing their median and tail KVS access latency down to 2.8 − 3.5 us and 5.4 − 7.8 us, respectively. Finally, we show that Dagger is beneficial for multi-tier end-to-end microservices with different threading models by evaluating it using an 8-tier application implementing a flight check-in service.

show abstract

Section: Discussionmentioning

confidence: 99%

Dagger: Towards Efficient RPCs in Cloud Microservices With Near-Memory Reconfigurable NICs

Lazarev

Adit

Xiang

et al. 2020

IEEE Comput. Arch. Lett.

View full text Add to dashboard Cite

show abstract

“…A software-defined control plane performs resource allocation and orchestration, and exploits the system flexibility to fulfill the resource needs of the applications (or virtual machines) running in the system to form a modular, vertically-integrated system [49]. A key outcome of the dRedBox is the IBM Thymesis Flow [35] that achieves similar resource and resource-management flexibility in mainstream IBM systems.…”

Section: The Dredbox Disaggregated Architecturementioning

confidence: 99%

“…Scalable Phylogeny Reconstruction with Disaggregated Near-memory Processing 25:3 Disaggregated computer architectures [33][34][35], which enable virtual machines to employ resources (CPUs, memory, accelerators) that are physically located on different servers, represent a promising solution to effectively meet memory requirements of future analyses, since an application can potentially deploy all memory resources in a data center. This, however, poses a major challenge to the efficient deployment of hardware acceleration because input/output data can reside on different servers than the ones hosting accelerators, hence requiring time-and energyconsuming, remote-data transfers that can diminish the gains of hardware acceleration.…”

Section: Introductionmentioning

confidence: 99%

Scalable Phylogeny Reconstruction with Disaggregated Near-memory Processing

Alachiotis

Skrimponis

Pissadakis

et al. 2021

ACM Trans. Reconfigurable Technol. Syst.

View full text Add to dashboard Cite

Disaggregated computer architectures eliminate resource fragmentation in next-generation datacenters by enabling virtual machines to employ resources such as CPUs, memory, and accelerators that are physically located on different servers. While this paves the way for highly compute- and/or memory-intensive applications to potentially deploy all CPUs and/or memory resources in a datacenter, it poses a major challenge to the efficient deployment of hardware accelerators: input/output data can reside on different servers than the ones hosting accelerator resources, thereby requiring time- and energy-consuming remote data transfers that diminish the gains of hardware acceleration. Targeting a disaggregated datacenter architecture similar to the IBM dReDBox disaggregated datacenter prototype, the present work explores the potential of deploying custom acceleration units adjacently to the disaggregated-memory controller on memory bricks (in dReDBox terminology), which is implemented on FPGA technology, to reduce data movement and improve performance and energy efficiency when reconstructing large phylogenies (evolutionary relationships among organisms). A fundamental computational kernel is the Phylogenetic Likelihood Function (PLF), which dominates the total execution time (up to 95%) of widely used maximum-likelihood methods. Numerous efforts to boost PLF performance over the years focused on accelerating computation; since the PLF is a data-intensive, memory-bound operation, performance remains limited by data movement, and memory disaggregation only exacerbates the problem. We describe two near-memory processing models, one that addresses the problem of workload distribution to memory bricks, which is particularly tailored toward larger genomes (e.g., plants and mammals), and one that reduces overall memory requirements through memory-side data interpolation transparently to the application, thereby allowing the phylogeny size to scale to a larger number of organisms without requiring additional memory.

show abstract

“…However, datacenters today suffer from low memory utilization (< 65%) [29,45,55,61], which results from imbalanced memory usages across a sea of servers. In response, academia and industry are working towards a new hardware architecture called memory disaggregation, where CPU and memory are physically separated into two network-attached components -compute servers and memory servers [16,23,30,36,38,39,42,49,55,56,65,71,78,79]. With memory disaggregation, CPU and memory can scale independently and different applications share a global disaggregated memory pool efficiently.…”

Section: Introductionmentioning

confidence: 99%

Sherman: A Write-Optimized Distributed B+Tree Index on Disaggregated Memory

Wang¹,

Lu²,

Shu³

2021

Preprint

View full text Add to dashboard Cite

Memory disaggregation architecture physically separates CPU and memory into independent components, which are connected via high-speed RDMA networks, greatly improving resource utilization of databases. However, such an architecture poses unique challenges to data indexing in databases due to limited RDMA semantics and near-zero computation power at memory-side. Existing indexes supporting disaggregated memory either suffer from low write performance, or require hardware modification.This paper presents Sherman, a write-optimized distributed B + Tree index on disaggregated memory that delivers high performance with commodity RDMA NICs. Sherman combines RDMA hardware features and RDMA-friendly software techniques to boost index write performance from three angles. First, to reduce round trips, Sherman coalesces dependent RDMA commands by leveraging in-order delivery property of RDMA. Second, to accelerate concurrent accesses, Sherman introduces a hierarchical lock that exploits on-chip memory of RDMA NICs. Finally, to mitigate write amplification, Sherman tailors the data structure layout of B + Tree with a two-level version mechanism. Our evaluation shows that, Sherman is one order of magnitude faster in terms of both throughput and 99th percentile latency on typical write-intensive workloads, compared with state-of-the-art designs.

show abstract

ThymesisFlow: A Software-Defined, HW/SW co-Designed Interconnect Stack for Rack-Scale Memory Disaggregation

Cited by 44 publications

References 23 publications

Dagger: Towards Efficient RPCs in Cloud Microservices With Near-Memory Reconfigurable NICs

Dagger: Towards Efficient RPCs in Cloud Microservices With Near-Memory Reconfigurable NICs

Scalable Phylogeny Reconstruction with Disaggregated Near-memory Processing

Sherman: A Write-Optimized Distributed B+Tree Index on Disaggregated Memory

Contact Info

Product

Resources

About