Scale-out processors

Abstract

Scale-out datacenters mandate high per-server throughput to get the maximum benefit from the large TCO investment. Emerging applications (e.g., data serving and web search) that run in these datacenters operate on vast datasets that are not accommodated by on-die caches of existing server chips. Large caches reduce the die area available for cores and lower performance through long access latency when instructions are fetched. Performance on scale-out workloads is maximized through a modestly-sized last-level cache that captures the instruction footprint at the lowest possible access latency. In this work, we introduce a methodology for designing scalable and efficient scale-out server processors. Based on a metric of performance-density, we facilitate the design of optimal multi-core configurations, called pods. Each pod is a complete server that tightly couples a number of cores to a small last-level cache using a fast interconnect. Replicating the pod to fill the die area yields processors which have optimal performance density, leading to maximum per-chip throughput. Moreover, as each pod is a stand-alone server, scale-out processors avoid the expense of global (i.e., interpod) interconnect and coherence. These features synergistically maximize throughput, lower design complexity, and improve technology scalability. In 20nm technology, scaleout chips improve throughput by 5x-6.5x over conventional and by 1.6x-1.9x over emerging tiled organizations.

A Case for Specialized Processors for Scale-Out Workloads

“…Conversely, the characteristics of scale-out workloads can be effectively leveraged to specialize processors for these workloads in order to gain area and energy efficiency in future servers. An example of such a specialized processor design that matches the needs of scaleout workloads is Scale-Out Processor, 4 which has been shown to improve the system throughput and the overall datacenter cost efficiency by almost an order of magnitude. 5 …”

“…Because the scale-out workloads perform only infrequent communication via the network, there is typically no read-write sharing in the applications; processors can therefore be designed as a collection of core islands using a low-bandwidth interconnect that does not enforce coherence between the islands, eliminating the power associated with the high-bandwidth interconnect as well as the power and area overheads of fine-grained coherence tracking. 4 Off-chip memory buses can be optimized for scale-out workloads by scaling back unnecessary bandwidth for systems with an insufficient number of cores. Memory controllers consume a large fraction of the chip area, and memory buses are responsible for a large fraction of the system power.…”

“…However, instead of taking a step backward and scaling back the memory bandwidth to match the requirements and throughput of conventional processors, a more effective solution would be to increase the processor throughput through specialization and thus utilize the available bandwidth resources. 4 T he impending plateau of voltage levels and a continued increase in chip density are forcing efficiency to be the primary driver of future processor designs. Our analysis shows that efficiently executing scaleout workloads requires optimizing the instruction-fetch path for multi-megabyte instruction working sets; reducing the core aggressiveness and LLC capacity to free area and power resources in favor of more cores, each with more hardware threads; and scaling back the overprovisioned on-chip and offchip bandwidth.…”

NOC-Out: Microarchitecting a Scale-Out Processor

“…An example of an existing scale-out processor design is the Oracle T-series, which features up to 16 cores, 3-6 MB LLC capacities, and a low-latency crossbar interconnect. Extending and formalizing the space of scale-out processors, researchers introduced the Scale-Out Processor (SOP) design methodology [15]. The SOP methodology, which provides an optimization framework for deriving optimal core counts and LLC capacities based on microarchitectural and technology parameters, advocates many cores, modestly-sized LLCs, and low interconnect delays.…”

Towards Near-Threshold Server Processors

“…The cores are modeled after Cortex-A57, a 3-way Out-of-Order (OoO) core, resembling those used in specialized many-cores for servers [7], [8]. Following the scale-out processor methodology [9], the chip is organized as a set of clusters, which exhibit an optimal ratio between core count and cache size. Although we calculate the optimal ratio as a 16-core cluster with a 4MB Last-Level Cache (LLC), we model 4-core clusters due to a lower simulation turnaround time.…”