Tofu Interconnect 2: System-on-Chip Integration of High-Performance Interconnect

Ajima, Yuichiro; Inoue, Takahiro; Hiramoto, Shinya; Uno, Shunji; Sumimoto, Shinji; Miura, Kiyotaka; Shida, Naoyuki; Kawashima, Takahiro; Okamoto, Takuji; Moriyama, Osamu; Ikeda, Yoshiro; Tabata, Takekazu; Yoshikawa, Toshio; Seki, Ken; Shimizu, Toshiyuki

doi:10.1007/978-3-319-07518-1_35

Cited by 30 publications

(10 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The direct network in many supercomputers usually adds additional interconnection dimensions for local communication. In K-computer (Ajima et al 2014), three dimensions are used for local rack communication in its Tofu network (Toyoshima 2010;Ajima et al 2012), and a similar torusinterconnected network is used in the 5D-Torus network of IBM Bluegene Q (Haring 2011). IBM PERCS (Arimilli et al 2010) uses the Alltoall interconnect design for local rack communication and a number of specific switches for connection among different racks, which is similar to the design of Dragonfly (Faanes et al 2012) proposed by Cray.…”

Section: Related Workmentioning

confidence: 99%

Wormhole optical network: a new architecture to solve long diameter problem in exascale computer

et al. 2019

View full text Add to dashboard Cite

The exascale computer will be built in the near future thanks to rapid innovations in semiconductor logic, memory, architectures, interconnections and other essential technologies. It is difficult to design an interconnection network that combines high performance with low power consumption. Therefore, building an interconnection network with high cost performance plays a critical role in building such a large scale system. Currently, torus-interconnected network like 6D-Torus possesses suitable properties for the petascale computer. However, the diameter within the torus-interconnected network is too long to achieve efficient global communication in the exascale computer. In addition, a direct connection method is not adaptive to the diverse characteristics of traffic. Here, we propose an architecture called Wormhole Optical Network (WON) for the exascale computer which is based on optical circuit switching. WON was designed to fully integrate into the electrical network of 6D-Torus. WON allows for the use of three novel technologies, namely the dynamic topology with optical links, algorithm of cross dimension order routing, and strategy of flow control for deadlock-free. We evaluated WON using both a prototype system and a simulator for the exascale computer. Our analysis shows that compared to the traditional electrical architecture, WON architecture reduced the time of data communication by 14-29% on exascale, a result obtained for a wide selection of diverse applications. Through enabling an SDN controller to adjust topology, WON maintains high utilization of optical links for inter-process communication from diverse applications. Further, we quantified WON's flexibility of job deployment for mitigating hotspot traffic. We show that WON reduced latency by 20-35% in the large-range deployment and improved throughput by 30% in the long-distance deployment.

show abstract

Section: Related Workmentioning

confidence: 99%

Wormhole optical network: a new architecture to solve long diameter problem in exascale computer

et al. 2019

View full text Add to dashboard Cite

show abstract

“…For examples, the Aries interconnect in Cray XC network series [6] with its dragonfly topology [7] is regarded as one of the best alternatives. The K computer [8] at the RIKEN Advanced Institute for Computational Science (AICS) in Kobe, Japan relies on a custom interconnect -Tofu Interconnect 2 [9] arranged in a 6D-Torus topology. One the main actors in Europe is Atos, that for HPC developed the BXI [10], Bull eXascale Interconnect.…”

Section: Introductionmentioning

confidence: 99%

EuroEXA Custom Switch: an innovative FPGA-based system for extreme scale computing in Europe

et al. 2020

View full text Add to dashboard Cite

EuroEXA is a major European FET research initiative that aims to deliver a proof-of-concept of a next generation Exa-scalable HPC platform. EuroEXA leverages on previous projects results (ExaNeSt, ExaNoDe and ECOSCALE) to design a medium scale but scalable, fully working HPC system prototype exploiting state-of-the-art FPGA devices that integrate compute accelerators and low-latency high-throughputnetwork. Exascale-class systems are expected to host a very large number of computing nodes, from 104 up to 105, so that capability and performances of the interconnect architecture are critical to achieve high computing efficiency at this scale. In this perspective, EuroEXA enhances the ExaNet architecture, inherited by the ExaNeSt project, and introduces a multi-tier, hybrid topology network built on top of an FPGA-integrated Custom Switch that provides high throughput and low inter-node traffic latency for the different layers of the network hierarchy. Deployment of a few testbeds is planned, with incremental complexity and equipped with complete software stack and runtime environment, to support the integration and test of the network design and to allow for evaluation of system performance and scalability through benchmarks based on real HPC applications. Design and integration activities are ongoing and the first small scale prototype (50 nodes) is expected to be completed in fall 2020 followed, one year later, by the deployment of the larger prototype (250/500 nodes).

show abstract

“…Among these topologies, the torus is popular because of its recursive and symmetric structure, simplicity, and scalability. For example, Fugaku adopted the 6-dimensional torus as the topology of its interconnection network [11].…”

Section: Introduction a Backgroundmentioning

confidence: 99%

Pairwise Disjoint Paths Routing in Tori

2020

View full text Add to dashboard Cite

The pairwise disjoint paths problem is to construct c disjoint paths s i d i (1 ≤ i ≤ c) between given pairs of nodes (s i , d i) (1 ≤ i ≤ c) in a graph whose connectivity is no less than 2c. It is a major problem for interconnection networks, together with the node-to-node disjoint paths problem, the node-to-set disjoint paths problem, and the set-to-set disjoint paths problem. In this paper, we propose an algorithm that solves this problem for a torus. In a previous work, Bossard and Kaneko have developed an algorithm that constructs disjoint paths between c pairs of nodes in a k-ary n-dimensional torus, where c ≤ n. The time complexity of their algorithm is O(c 4 n + kcn), and the maximum path length is k/2 n + 2k(c − 1). However, the algorithm proposed in this paper achieves a time complexity of O(c 3 n + kcn), and its maximum path length is k/2 n + (3k/2 − 2)(c − 1), which are both improvements over the previous algorithm. We also conducted an evaluation experiment to show that the average path lengths are proportional to k if n is fixed, and to n if k is fixed. The theoretical maximum path lengths were not attained by the paths constructed by our algorithm, and the average execution time was proportional to k if n is fixed, and to n 2 if k is fixed. Additional experimental results show that, compared to the previous algorithm by Bossard and Kaneko, our algorithm achieves a better performance with respect to the maximum path lengths, but both algorithms achieve a similar level of performance with respect to the average maximum path lengths. Also, it is shown that the average execution time of our algorithm is about O(n 2), which is better than the average execution time of the previous algorithm O(n 3) in the experimental framework.

show abstract

Tofu Interconnect 2: System-on-Chip Integration of High-Performance Interconnect

Cited by 30 publications

References 4 publications

Wormhole optical network: a new architecture to solve long diameter problem in exascale computer

Wormhole optical network: a new architecture to solve long diameter problem in exascale computer

EuroEXA Custom Switch: an innovative FPGA-based system for extreme scale computing in Europe

Pairwise Disjoint Paths Routing in Tori

Contact Info

Product

Resources

About