swDNN: A Library for Accelerating Deep Learning Applications on Sunway TaihuLight

Fang, Jiayuan; Fu, Haohuan; Zhao, Wenlai; Chen, Bingwei; Zheng, Weijie; Yang, Guangwen

doi:10.1109/ipdps.2017.20

Cited by 71 publications

(38 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Optimizations on Sunway architecture. There're a few works exploiting architectural features on Sunway, e.g., heterogeneous computing cores, SIMD, register-level communication, SPM, and so on, which are either hand-tuned application-specific implementations [3,8,19,61], or domain-specific frameworks [18,36,75]. Specially, [38,62,76] perform hand-tuned tiling for parallelism.…”

Section: Related Workmentioning

confidence: 99%

Bandwidth-Aware Loop Tiling for DMA-Supported Scratchpad Memory

Liu

Cui

et al. 2020

Proceedings of the ACM International Conference on Parallel Architectures and Compilation Techniques

View full text Add to dashboard Cite

Scratchpad Memory (SPM) is widely used in emerging domain-specific architectures and accelerators for improving energy efficiency and time predictability. Typically, SPM-based architectures use DMA for fetching data from off-chip memory and global load instructions for loading fine-grained data directly into registers. For such architectures, neither capacity-only nor bandwidthonly loop tiling can efficiently use the bandwidth and SPM. This paper introduces a bandwidth-aware loop tiling approach that enables a tradeoff between SPM space utilization and bandwidth utilization to be made, by leveraging a runtime tiling framework and a cross-host-kernel IPA. Experimental results demonstrate that our approach can achieve the performance improvement of up to 4x, with a geometric average of 26%.

show abstract

Section: Related Workmentioning

confidence: 99%

Bandwidth-Aware Loop Tiling for DMA-Supported Scratchpad Memory

Liu

Cui

et al. 2020

Proceedings of the ACM International Conference on Parallel Architectures and Compilation Techniques

View full text Add to dashboard Cite

show abstract

“…Each CPE has 16 KB L1 instruction cache and 64 KB local data memory (LDM) which can be configured as user-controlled fast buffer. A performance model based on the three-level (REG-LDM-MEM) memory hierarchy was proposed (Fang et al, 2017). The CPE can either directly access the global memory with a limited bandwidth of 8GB/s, or through a REG-LDM-MEM memory hierarchy to obtain much higher bandwidth.…”

Section: The Sw26010 Processormentioning

confidence: 99%

“…The Sunway Taihulight is a Chinese home-grown supercomputing system with 40960 compute nodes, providing a peak performance of 125Pflops and a sustained Linpack performance of 93PFlops, and ever ranked No.1 on the Top-500 supercomputers in the past two years. There are abundant studies on this platform containing atmospheric dynamics, earthquake simulation, deep learning and molecular dynamics and two of them won the ACM Gorden Bell prizes in 2016 and 2017 (Yang, 2016;Fu et al, 2017;Fang et al, 2017;Duan, 2018).…”

Section: Introductionmentioning

confidence: 99%

Towards a Peta-scale Unstructured Computational Fluid Dynamics (CFD) Acceleration Toolkit based on Sunway architectures?

Liu¹,

Ren²,

Wang³

et al. 2019

Proceedings of Global Power &Amp; Propulsion Society

Self Cite

View full text Add to dashboard Cite

Adopting Large Eddy Simulation (LES) to simulate the complex flow is appropriate to overcome the limitation of current Reynolds-Averaged Navier-Stokes (RANS) modeling and it provides a deeper understanding of the complicated transitional and turbulent flow mechanism. However, the large computational cost limits its application in high Reynolds number flow. Though Sunway TaihuLight supercomputer has revealed its remarkable performance on atmospheric dynamics and earthquake simulation, there are still some unsolved problems for researchers: unstructured mesh and complicated programming pattern on SW26010. Therefore, we develop an ultra-scalable Unstructured Acceleration Toolkit (UNAT). In this toolkit, we derive a graph decomposition model of computing flow and reuse data through L1 cache to optimize the operator. Multi-level block method and register communication are developed to achieve the best performance of CPEs. CFD engineer can ignore the detail of parallel programming on SW26010 and utilize the operator interfaces offered by UNAT. Finally, we deploy UNAT to accelerate a combustion code and obtain 12-19x performance speed-up on specific operators, onto 65 cores within the chip compared with MPE.

show abstract

“…Compared with recent research on regular problems on Sunway architecture, such as stencil [3], DNN [12], GEMM [18,27] and fully-implicit solver for nonhydrostatic atmospheric dynamics [66], our work presented in this paper is more complicated, as the irregularities from various matrix sparsity structures are dynamic and leveraging such irregularity is known to be more challenging [57,67].…”

Section: Related Workmentioning

confidence: 99%

swSpTRSV

Wang

Liu

Xue

et al. 2018

Proceedings of the 23rd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming

View full text Add to dashboard Cite

Sparse triangular solve (SpTRSV) is one of the most important kernels in many real-world applications. Currently, much research on parallel SpTRSV focuses on level-set construction for reducing the number of inter-level synchronizations. However, the out-of-control data reuse and high cost for global memory or shared cache access in inter-level synchronization have been largely neglected in existing work. In this paper, we propose a novel data layout called Sparse Level Tile to make all data reuse under control, and design a Producer-Consumer pairing method to make any inter-level synchronization only happen in very fast register communication. We implement our data layout and algorithms on an SW26010 many-core processor, which is the main buildingblock of the current world fastest supercomputer Sunway Taihulight. The experimental results of testing all 2057 square matrices from the Florida Matrix Collection show that our method achieves an average speedup of 6.9 and the best speedup of 38.5 over parallel level-set method. Our method also outperforms the latest methods on a KNC many-core processor in 1856 matrices and the latest methods on a K80 GPU in 1672 matrices, respectively.

show abstract

swDNN: A Library for Accelerating Deep Learning Applications on Sunway TaihuLight

Cited by 71 publications

References 7 publications

Bandwidth-Aware Loop Tiling for DMA-Supported Scratchpad Memory

Bandwidth-Aware Loop Tiling for DMA-Supported Scratchpad Memory

Towards a Peta-scale Unstructured Computational Fluid Dynamics (CFD) Acceleration Toolkit based on Sunway architectures?

swSpTRSV

Contact Info

Product

Resources

About