Blue Gene/L compute chip: Memory and Ethernet subsystem

Abstract

The Blue Genet/L compute chip is a dual-processor system-on-a-chip capable of delivering an arithmetic peak performance of 5.6 gigaflops. To match the memory speed to the high compute performance, the system implements an aggressive three-level on-chip cache hierarchy. The implemented hierarchy offers high bandwidth and integrated prefetching on cache hierarchy levels 2 and 3 (L2 and L3) to reduce memory access time. A Gigabit Ethernet interface driven by direct memory access (DMA) is integrated in the cache hierarchy, requiring only an external physical link layer chip to connect to the media. The integrated L3 cache stores a total of 4 MB of data, using multibank embedded dynamic random access memory (DRAM). The 1,024-bit-wide data port of the embedded DRAM provides 22.4 GB/s bandwidth to serve the speculative prefetching demands of the two processor cores and the Gigabit Ethernet DMA engine. To reduce hardware overhead due to cache coherence intervention requests, memory coherence is maintained by software. This is particularly efficient for regular highly parallel applications with partitionable working sets. The system further integrates an on-chip double-data-rate (DDR) DRAM controller for direct attachment of main memory modules to optimize overall memory performance and cost. For booting the system and low-latency interprocessor communication and synchronization, a 16-KB static random access memory (SRAM) and hardware locks have been added to the design. IBM J. RES. & DEV. VOL. 49 NO. 2/3 MARCH/MAY 2005 M. OHMACHT ET AL.

Integrated network interfaces for high-bandwidth TCP/IP

“…Although not found in current high-performance server systems, CPUs with integrated Ethernet NICs appear in other environments [8,33], are rumored to be present in some upcoming servers [13], and have been shown to provide substantial performance improvements on server workloads [5]. (See Section 2.1 for further discussion.)…”

Global management of cache hierarchies

“…The ultimate goal, of course, is to satisfy most of the references from L1. Keeping the L1 size fixed and assuming it is not fully associative (although there exist some examples of fully associative L1s in real machines [24]), we try to decrease conflict misses by reducing contention among L1 sets. We do so by keeping some blocks out of the L1.…”

Impact of multicores on large-scale molecular dynamics simulations

“…The experimental data collection for this study is undertaken on x86-based standalone systems, Cray XT3 [9], Cray XT4 [10], IBM Blue Gene/L [15], and Blue Gene/P [4] systems. The x86-based standalone systems include an eight socket, dual-core AMD Opteron 8216 [2], an eight socket, quad-core AMD Opteron 8350 [1], and an Intel quad-core Clovertown system [18].…”