HyperPlane: A Scalable Low-Latency Notification Accelerator for Software Data Planes

Mirhosseini, Amirhossein; Golestani, Hossein Bakhshi; Wenisch, Thomas F.

doi:10.1109/micro50266.2020.00074

Cited by 11 publications

(3 citation statements)

References 64 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Conceptually and semantically, cpoll is similar to MWAIT in the x86 architecture [64], QWAIT in HyperPlane [111], and PCIe's lightweight notification (LN) proposal [50,136]. Nonetheless, cpoll differs from them because it is designed to be portable and platform/CPU-agnostic for off-chip devices.…”

Section: B Coherence-assisted Accelerator Notificationmentioning

confidence: 99%

See 1 more Smart Citation

ORCA: A Network and Architecture Co-design for Offloading us-scale Datacenter Applications

Yang¹,

Huang²,

Sun³

et al. 2022

Preprint

View full text Add to dashboard Cite

Responding to the "datacenter tax" and "killer microseconds" problems for datacenter applications, diverse solutions including Smart NIC-based ones have been proposed. Nonetheless, they often suffer from high overhead of communications over network and/or PCIe links. To tackle the limitations of the current solutions, this paper proposes ORCA, a holistic network and architecture co-design solution that leverages current RDMA and emerging cache-coherent off-chip interconnect technologies. Specifically, ORCA consists of four hardware and software components: (1) unified abstraction of inter-and intra-machine communications managed by one-sided RDMA write and cache-coherent memory write; (2) efficient notification of requests to accelerators assisted by cache coherence; (3) cache-coherent accelerator architecture directly processing requests received by NIC; and (4) adaptive device-tohost data transfer for modern server memory systems consisting of both DRAM and NVM exploiting state-of-the-art features in CPUs and PCIe. We prototype ORCA with a commercial system and evaluate three popular datacenter applications: in-memory key-value store, chain replication-based distributed transaction system, and deep learning recommendation model inference. The evaluation shows that ORCA provides 30.1∼69.1% lower latency, up to 2.5× higher throughput, and ∼ 3× higher power efficiency than the current state-of-the-art solutions.

show abstract

Section: B Coherence-assisted Accelerator Notificationmentioning

confidence: 99%

“…Hence, there will be no scalability concern for the cpoll mechanism. Even if the buffers are not allocated consecutively in the memory, the overhead of address lookup should not be a concern as well (to O(1K) level buffers at least), as demonstrated by HyperPlane [111].…”

Section: B Coherence-assisted Accelerator Notificationmentioning

confidence: 99%

ORCA: A Network and Architecture Co-design for Offloading us-scale Datacenter Applications

Yang¹,

Huang²,

Sun³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Microservices exhibit mean service times ranging from tens of microseconds to single-digit seconds [41]. Whereas implementing a shared queue across cores can be challenging for short s-scale microservices, recent frameworks have sought to mitigate its overheads in software [24,37] or hardware [12,32,43].…”

Section: Scheduling Within An Instancementioning

confidence: 99%

μSteal

Mirhosseini

Wenisch

2021

Proceedings of the ACM International Conference on Supercomputing

Self Cite

View full text Add to dashboard Cite

Modern internet services are moving towards distributed microservice architectures, wherein a complex application is decomposed into numerous discrete microservices to improve programmability, reliability, manageability, and scalability. A key property of microservice-based architectures is that common microservices may be shared by multiple end-to-end cloud services. As an example, a speech-recognition microservice might serve as an early node in the microservice graphs of several end-to-end services. However, given the dissimilarities across microservice graphs and varying end-to-end latency constraints across services, shared microservices may need to operate under differing latency constraints for each service. As a result, in existing systems, most providers either deploy multiple instance pools for each latency constraint, or require all requests to needlessly meet the most stringent constraint.In this paper, we argue that sharing microservice instances across multiple services can reduce significantly the number of instances, especially under highly asymmetric latency constraints. We propose a request scheduling mechanism, called Steal, which leverages preemptive work and resource stealing to schedule the arriving requests to cores within a "mixed-criticality" microservice instance.Steal provisions "core reservations" for each request class based on their latency requirements, but allows a class to steal cores from other classes if they would otherwise remain idle. But, when a class requires its full reservation, Steal preempts stolen cores, returning them to their reserved class. Steal employs a runtime feedback controller augmented by a queuing theory-based analytical model to tune core reservations across classes, seeking to maximize the request throughput within each instance while meeting all classes' latency constraints. We show that Steal reduces required instances for several shared microservice deployments by 1.29× as compared to deploying multiple, segregated instance pools. CCS CONCEPTS• Computer systems organizations → Multi-core architectures; • Network → Cloud computing.

show abstract

Rambda: RDMA-driven Acceleration Framework for Memory-intensive µs-scale Datacenter Applications

Yang

Huang

Sun

et al. 2023

2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA)

View full text Add to dashboard Cite

HyperPlane: A Scalable Low-Latency Notification Accelerator for Software Data Planes

Cited by 11 publications

References 64 publications

ORCA: A Network and Architecture Co-design for Offloading us-scale Datacenter Applications

ORCA: A Network and Architecture Co-design for Offloading us-scale Datacenter Applications

μSteal

Rambda: RDMA-driven Acceleration Framework for Memory-intensive µs-scale Datacenter Applications

Contact Info

Product

Resources

About