On-Chip Communication Network for Efficient Training of Deep Convolutional Networks on Heterogeneous Manycore Systems

Abstract

Convolutional Neural Networks (CNNs) have shown a great deal of success in diverse application domains including computer vision, speech recognition, and natural language processing. However, as the size of datasets and the depth of neural network architectures continue to grow, it is imperative to design high-performance and energy-efficient computing hardware for training CNNs. In this paper, we consider the problem of designing specialized CPU-GPU based heterogeneous manycore systems for energy-efficient training of CNNs. It has already been shown that the typical on-chip communication infrastructures employed in conventional CPU-GPU based heterogeneous manycore platforms are unable to handle both CPU and GPU communication requirements efficiently. To address this issue, we first analyze the on-chip traffic patterns that arise from the computational processes associated with training two deep CNN architectures, namely, LeNet and CDBNet, to perform image classification. By leveraging this knowledge, we design a hybrid Network-on-Chip (NoC) architecture, which consists of both wireline and wireless links, to improve the performance of CPU-GPU based heterogeneous manycore platforms running the above-mentioned CNN training workloads. The proposed NoC achieves 1.8× reduction in network latency and improves the network throughput by a factor of 2.2 for training CNNs, when compared to a highly-optimized wireline mesh NoC. For the considered CNN workloads, these network-level improvements translate into 25% savings in full-system energy-delay-product (EDP). This demonstrates that the proposed hybrid NoC for heterogeneous manycore architectures is capable of significantly accelerating training of CNNs while remaining energy-efficient.

Learning-Based Application-Agnostic 3D NoC Design for Heterogeneous Manycore Systems

“…Due to its heterogeneity, CPU-GPU based systems exhibit several interesting traffic characteristics, for instance, GPUs typically only communicate with a few shared last level caches (LLCs) which results in many-to-few traffic patterns (i.e., many GPUs communicate with a few LLCs) with negligible inter-GPU communication [7], [13], [14]. This can cause the LLCs to become bandwidth bottlenecks under heavy network loads and lead to significant performance degradation [7]. In addition, since heterogeneous systems share the memory resources, the GPUs can monopolize the memory and cause high CPU memory access latency [15].…”

“…Due to the heterogeneity of the cores integrated on a single chip, the communication requirements for each core can vary significantly. For example, in a CPU-GPU based heterogeneous system, CPUs require low memory latency while GPUs need high-throughput data transfers [7]. In addition to the individual core requirements, 3D ICs allow dense circuit integration but have much higher power density than their 2D counterparts.…”

Engineer the Channel and Adapt to it: Enabling Wireless Intra-Chip Communication

“…This does not invalidate the potential of the WNoC paradigm, but leads to erroneous assumptions on the achievable speed and power. For instance, many WNoC architectures assume rates over 10 Gb/s [12], [27], [28], which may not be achievable due to multipath effects. Other works obtain power consumption estimates by assuming path losses between 25 and 30 dB [36]- [39], values that are far from the true in standard chip packages.…”

Ring-mesh: a scalable and high-performance approach for manycore accelerators

“…Figures 12,13,14 show the achieved network throughput for all three traffic patterns. Similar to latency, the network throughput is also consistent with the packet injection rate.…”