Power, programmability, and granularity: The challenges of ExaScale computing

Abstract

Reaching an ExaScale computer by the end of the decade, and enabling the continued performance scaling of smaller systems requires significant research breakthroughs in three key areas: power efficiency, programmability, and execution granularity. To build an ExaScale machine in a power budget of 20 MW requires a 200-fold improvement in energy per instruction: from 2 nJ to 10 pJ. Only 4x is expected from improved technology. The remaining 50x must come from improvements in architecture and circuits. To program a machine of this scale requires more productive parallel programming environments-that make parallel programming as easy as sequential programming is today. Finally, problem size and memory size constraints prevent the continued use of weak scaling, requiring these machines to extract parallelism at very fine granularity-down to the level of a few instructions. This talk discusses these challenges and current approaches to address them. About the SpeakerBill is the Willard R. and Inez Kerr Bell Professor of Engineering at Stanford University and Chief Scientist at NVIDIA Corporation. Bill and his group have developed system architecture, network architecture, signaling, routing, and synchronization technology that can be found in most large parallel computers today. Bill has worked with Cray Research and Intel to incorporate his innovations in commercial parallel computers, with Avici Systems to incorporate his technology into Internet routers, co-founded Velio Communications to commercialize high-speed signaling technology, and co-founded Stream Processors, Inc. to commercialize stream processor technology. He is a Member of the National Academy of Engineering, a Fellow of the IEEE, a Fellow of the ACM, and a Fellow of the American Academy of Arts and Sciences. He has received numerous honors including the ACM Eckert-Mauchly Award, the IEEE Seymour Cray Award, and the ACM Maurice Wilkes award. He currently leads projects on computer architecture, network architecture, and programming systems. He has published over 200 papers in these areas, holds over 75 issued patents, and is an author of the textbooks, Digital Systems Engineering and Principles and Practices of Interconnection Networks.

GreenBST: Energy-Efficient Concurrent Search Tree

“…For a vEB tree with the same height, the required block transfers is only two. As shown in Figure 2b, locating the key in leaf-node 12 requires only a transfer of nodes (1, 2, 3), followed by a transfer of nodes (10,11,12). Generally, the data transfer (or I/O) complexity of searching for a key in a tree of size N is now reduced to log 2 N log 2 B = log B N , simply by using an efficient tree layout so that nearby nodes are located in adjacent memory locations.…”

“…Recent researches have suggested that the energy consumption of future computing systems will be dominated by the cost of data movement [12,34,35]. It is predicted that for 10nm technology chips, the energy required between accessing data in nearby on-chip memory and accessing data across the chip, will differ as much as 75× (2pJ versus 150pJ), whereas the energy required between accessing on-chip data and accessing off-chip data will only differ 2× (150pJ versus 300pJ) [12].…”

“…It is predicted that for 10nm technology chips, the energy required between accessing data in nearby on-chip memory and accessing data across the chip, will differ as much as 75× (2pJ versus 150pJ), whereas the energy required between accessing on-chip data and accessing off-chip data will only differ 2× (150pJ versus 300pJ) [12]. Therefore, in order to construct energy-efficient software systems, data structures and algorithms must not only be concerned with whether the data is on-chip (e.g., in cache) or not (e.g., in DRAM), but must consider also data locality in finer-granularity: where the data is located on the chip.…”

Energy Efficiency Improvement by Dynamic Reconfiguration for Embedded Systems

“…The reduction of power consumption has become an important problem in not only super computers but also electronic devices [1], [2]. For example, the concept, dark silicon, comes from the gap between Moore's Law [3] and Dennard scaling [4]; the shortage of LSI power disables a larger fraction of circuits whenever process rule advances.…”

Finite Element Algorithms and Data Structures on Graphical Processing Units

“…The evolution trend of computing architectures shows an increasing gap between computational performance and bandwidth to off-chip memory [8]; it is already apparent that while GPUs have a theoretical compute performance ten times higher than CPUs, the bandwidth to off-chip DRAM is only 3-5 times faster. Also, with an increasing number of cores on a single chip the amount of on-chip memory per core decreases.…”