The PERCS High-Performance Interconnect

Arimilli, Baba; Arimilli, Ravi; Chung, Vicente; Clark, Scott; Denzel, Wolfgang E.; Drerup, Ben; Hoefler, Torsten; Joyner, Jody; Lewis, Jerry M.; Li, Jian; Ni, Nan; Rajamony, Ram

doi:10.1109/hoti.2010.16

Cited by 172 publications

(133 citation statements)

References 2 publications

Supporting

Mentioning

125

Contrasting

Order By: Relevance

“…Two prominent implementations of multi-level direct networks are the PERCS interconnect by IBM [3] and the Cascade system by Cray [4]. We focus on the Cascade system which is based on the dragonfly topology designed by Kim et al [1].…”

Section: The Dragonfly Interconnectmentioning

confidence: 99%

“…Multi-level direct networks have been proposed recently by several researchers as a scalable topology for connecting a large number of nodes together [1], [2], [3], [4]. The basic idea behind these networks is to have a topology that resembles an all-to-all at each level of the hierarchy which gives the impression of a highly connected network.…”

Section: The Dragonfly Interconnectmentioning

confidence: 99%

“…This has pushed the HPC community towards designing low-diameter fast networks with large bisection bandwidth. The dragonfly topology [1] and its variants [2], [3], [4] are actively being explored as the interconnects that satisfy all these requirements.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Maximizing Throughput on a Dragonfly Network

Jain

Bhatelé

et al. 2014

SC14: International Conference for High Performance Computing, Networking, Storage and Analysis

View full text Add to dashboard Cite

Abstract-Interconnection networks are a critical resource for large supercomputers. The dragonfly topology, which provides a low network diameter and large bisection bandwidth, is being explored as a promising option for building multi-Petaflop/s and Exaflop/s systems. Unlike the extensively studied torus networks, the best choices of message routing and job placement strategies for the dragonfly topology are not well understood. This paper aims at analyzing the behavior of an interconnect based on the dragonfly topology for various routing strategies, job placement policies, and application communication patterns. Our study is based on a novel model that predicts traffic on individual links for direct, indirect, and adaptive routing strategies. We analyze results for individual communication patterns and some common parallel job workloads. The predictions presented in this paper are for a 100+ Petaflop/s prototype machine with 92,160 highradix routers and 8.8 million cores.

show abstract

Section: The Dragonfly Interconnectmentioning

confidence: 99%

Section: The Dragonfly Interconnectmentioning

confidence: 99%

See 1 more Smart Citation

Maximizing Throughput on a Dragonfly Network

Jain

Bhatelé

et al. 2014

SC14: International Conference for High Performance Computing, Networking, Storage and Analysis

View full text Add to dashboard Cite

show abstract

“…PERCS is a supercomputer design by IBM that uses POWER7 processors and a two-level directly-connected network [10]. This was the intended design for the Blue Waters system at Illinois, however, the plans changed later.…”

Section: The Percs Architecturementioning

confidence: 99%

Simulation-Based Performance Analysis and Tuning for a Two-Level Directly Connected System

Totoni

Bhatele

Bohm

et al. 2011

2011 IEEE 17th International Conference on Parallel and Distributed Systems

View full text Add to dashboard Cite

Abstract-Hardware and software co-design is becoming increasingly important due to complexities in supercomputing architectures. Simulating applications before there is access to the real hardware can assist machine architects in making better design decisions that can optimize application performance. At the same time, the application and runtime can be optimized and tuned beforehand. BigSim is a simulation-based performance prediction framework designed for these purposes. It can be used to perform packet-level network simulations of parallel applications using existing parallel machines. In this paper, we demonstrate the utility of BigSim in analyzing and optimizing parallel application performance for future systems based on the PERCS network. We present simulation studies using benchmarks and real applications expected to run on future supercomputers. Future petascale systems will have more than 100,000 cores, and we present simulations at that scale.

show abstract

“…The topologies of the local and global interconnects are typically low-diameter direct topologies that exploit highradix routers. For example, the IBM PERCS Interconnect [2] employs an all-to-all topology (1D-Flattened Butterfly) in both the local and global interconnects, while the Cray XC30 (codenamed "Cascade", [3]) employs an 1D-Flattened Butterfly (FB) for the global interconnect and a 2D-FB within groups.…”

mentioning

confidence: 99%

OFAR-CM: Efficient Dragonfly Networks with Simple Congestion Management

García

Vallejo

Beivide

et al. 2013

2013 IEEE 21st Annual Symposium on High-Performance Interconnects

View full text Add to dashboard Cite

Abstract-Dragonfly networks are appealing topologies for large-scale Datacenter and HPC networks, that provide high throughput with low diameter and moderate cost. However, they are prone to congestion under certain frequent traffic patterns that saturate specific network links. Adaptive nonminimal routing can be used to avoid such congestion. That kind of routing employs longer paths to circumvent local or global congested links. However, if a distance-based deadlock avoidance mechanism is employed, more Virtual Channels (VCs) are required, what increases design complexity and cost.OFAR (On-the-Fly Adaptive Routing) is a previously proposed routing that decouples VCs from deadlock avoidance, making local and global misrouting affordable. However, the severity of congestion with OFAR is higher, as it relies on an escape subnetwork with low bisection bandwidth. Additionally, OFAR allows for unlimited misroutings on the escape subnetwork, leading to unbounded paths in the network and long latencies.In this paper we propose and evaluate OFAR-CM, a variant of OFAR combined with a simple congestion management (CM) mechanism which only relies on local information, specifically the credit count of the output ports in the local router. With simple escape subnetworks such as a Hamiltonian ring or a tree, OFAR outperforms former proposals with distance-based deadlock avoidance. Additionally, although long paths are allowed in theory, in practice packets arrive at their destination in a small number of hops. Altogether, OFAR-CM constitutes the first practicable mechanism to the date that supports both local and global misrouting in Dragonfly networks.

show abstract

The PERCS High-Performance Interconnect

Cited by 172 publications

References 2 publications

Maximizing Throughput on a Dragonfly Network

Maximizing Throughput on a Dragonfly Network

Simulation-Based Performance Analysis and Tuning for a Two-Level Directly Connected System

OFAR-CM: Efficient Dragonfly Networks with Simple Congestion Management

Contact Info

Product

Resources

About