SoftSKU

Sriraman, Akshitha; Dhanotia, Abhishek; Wenisch, Thomas F.

doi:10.1145/3307650.3322227

Cited by 66 publications

(6 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Existing benchmarks that are generally used to design processors do not accurately represent the workloads at hyperscalars. As highlighted by several previous works [3,5,20,36,35], cloud workloads exhibit fundamentally different IPC, cache miss rates, and other metrics compared to standard benchmarks. These factors make the SPEC and other commercially available benchmarks a bad proxy for studying the performance of server processors in a datacenter.…”

Section: Code Memory Bw and Latency Challenges In Cloud Datacen-tersmentioning

confidence: 97%

“…SoftSKU [36] examines Meta's cloud workloads, understanding OS and hardware behavior using "Soft SKUs" by adjusting core/uncore frequencies, LLC prioritization, and active core counts. They observe similar IPC, cache, and TLB MPKIs.…”

Section: Related Workmentioning

confidence: 99%

“…the fleet-wide 1 IPC (instructions per cycle) for major hyperscalers is significantly lower than that of the benchmark suites [36,20]. With millions of servers running these workloads, even small improvements in IPC from understanding the behavior of these workloads can result in significant cost savings and efficiency improvements across the fleet.…”

Section: Introductionmentioning

confidence: 99%

“…The first source of stalls, code fetch-related stalls are a significant problem in cloud workloads [3,36]. Using Mem-Prof, we observe that workloads at Meta run similar code and share page table mappings across cores.…”

Section: Introductionmentioning

confidence: 99%

“…This paper is the first to study how cloud workloads interact with the memory system and to use these observations to propose shared µ-arch structures and memory bandwidth tiering. While previous works from Google [3,20] and Meta [36,35] have investigated the µ-arch behavior of hyperscale workloads, they have not performed detailed studies of the code behavior, memory bandwidth distribution, or hardware prefetcher efficiency. In summary, this paper makes the following contributions:…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Workload Behavior Driven Memory Subsystem Design for Hyperscale

Mahar¹,

Wang²,

Shu³

et al. 2023

Preprint

View full text Add to dashboard Cite

Hyperscalars run services across a large fleet of servers, serving billions of users worldwide. However, these services exhibit different behaviors compared to commonly available benchmark suites, leading to server architectures that are suboptimal for cloud workloads. As datacenters emerge as the primary server processor market, optimizing server processors for cloud workloads by better understanding their behavior is an area of interest. To address this, we present MemProf, a memory profiler that profiles the three major reasons for stalls in cloud workloads: code-fetch, memory bandwidth, and memory latency. We use MemProf to understand the behavior of cloud workloads at Meta and propose and evaluate micro-architectural and memory system design improvements that help cloud workloads' performance.MemProf's code analysis shows that cloud workloads at Meta execute the same code across CPU cores. Using this, we propose shared micro-architectural structures-a shared L2 I-TLB and a shared L2 cache. Next, to help with memory bandwidth stalls, using workloads' memory bandwidth distribution, we find that only a few pages contribute to most of the system bandwidth. We use this finding to evaluate a new high-bandwidth, small-capacity memory tier and show that it performs 1.46× better than the current baseline configuration. Finally, we look into ways to improve memory latency for cloud workloads. Profiling using MemProf reveals that L2 hardware prefetchers, which are commonly used to reduce memory latency, have very low coverage and consume a significant amount of memory bandwidth. To help improve future hardware prefetcher performance, we built an efficient memory tracing tool to collect and validate production memory access traces. Our memory tracing tool adds significantly less overhead than DynamoRIO, enabling tracing production workloads.

show abstract