An Overview of Efficient Interconnection Networks for Deep Neural Network Accelerators

Nabavinejad, Seyed Morteza; Baharloo, Mohammad; Chen, Kun-Chih; Palesi, Maurizio; Kogel, Tim; Ebrahimi, Masoumeh

doi:10.1109/jetcas.2020.3022920

Cited by 54 publications

(15 citation statements)

References 127 publications

(110 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These accelerators employ different computing cores such as ASICs [12,13,39], FPGAs [43,69,72], and GPUs [44,55,67] or different computing paradigms such as processing in/nearmemory [4,14,22,36,63] for accelerating DNN inference. GPU accelerators are a favorable choice for DNN accelerators due to their programmability and scalability features [49].…”

Section: Related Workmentioning

confidence: 99%

Coordinated Batching and DVFS for DNN Inference on GPU Accelerators

Nabavinejad

Reda

Ebrahimi

2022

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

Deployment of real-time ML services on warehouse-scale infrastructures is on the increase. Therefore, decreasing latency and increasing throughput of deep neural network (DNN) inference applications that empower those services have attracted attention from both academia and industry. A common solution to address this challenge is leveraging hardware accelerators such as GPUs. To improve the inference throughput of DNNs deployed on GPU accelerators, two common approaches are employed: Batching and Multi-Tenancy. Our preliminary experiments show that the effect of these approaches on the throughput depends on the DNN architecture. Taking this observation into account, we design and implement DNNScaler which aims to maximize the throughput of interactive AI-powered services while meeting their latency requirements. DNNScaler first detects the suitable approach (Batching or Multi-Tenancy) that would be most beneficial for a DNN regarding throughput improvement. Then, it adjusts the control knob of the detected approach (batch size for Batching and number of co-located instances for Multi-Tenancy) to maintain the latency while increasing the throughput. Conducting an extensive set of experiments using well-known DNNs from a variety of domains, several popular datasets, and a cutting-edge GPU, the results indicate that DNNScaler can improve the throughput by up to 14x (218% on average) compared with the previously proposed approach, while meeting the latency requirements of the services.

show abstract

Section: Related Workmentioning

confidence: 99%

Coordinated Batching and DVFS for DNN Inference on GPU Accelerators

Nabavinejad

Reda

Ebrahimi

2022

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

show abstract

“…Compared with a conventional neural network, CNN is more suitable for use in images; convolution corresponds to the local features of the image, and pooling makes the feature obtained by convolution spatially invariant. A convolutional neural network that is quite efficient for handwritten character recognition is proposed, the originator of many current convolutional neural networks [ 22 ]. As shown in Figure 1 , the Le Net network consists entirely of a convolutional layer and a fully connected layer.…”

Section: Model Construction Of Deep Neural Networkmentioning

confidence: 99%

A Study on the Relationship between Painter’s Psychology and Anime Creation Style Based on a Deep Neural Network

Chen

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

We conduct an in-depth study on the model construction of deep neural networks and design a model of painter’s psychology and anime creation style to realize the study of the relationship between painter’s psychology and anime creation style based on deep neural networks. This paper proposes an animation creation psychology classification algorithm that integrates human cognitive deep network structure optimization. The algorithm analyzes the connection between different convolutional layer features and animation characteristics through animation creation style CNN feature visualization. It interactively uses the knowledge of animation creation psychology expression techniques to optimize the network structure. This paper proposes a scene animation network based on spectral difference perception style. By analyzing the characteristics and differences in the spectrum between realistic and anime domain images, the generator is guided to learn the mapping relationships better to fit the style distribution of anime domain images. This paper uses a fully convolutional structure; the network is more lightweight and supports image inputs of arbitrary size, which can keep the semantic system of the background unchanged while highly deforming the five facial features, moving toward the goal of human-scene fusion for the animation task.

show abstract

“…In contrast, our platform is topology-agnostic and adheres to an industry-standard protocol. An overview of on-and off-chip interconnects for NN accelerators is presented in [26]. They highlight the need for non-mesh topologies in NN accelerators, to which we contribute with our case study and topologyindependent platform.…”

Section: Related Workmentioning

confidence: 99%

An Open-Source Platform for High-Performance Non-Coherent On-Chip Communication

Kurth¹,

Rönninger²,

Benz³

et al. 2021

IEEE Trans. Comput.

View full text Add to dashboard Cite

On-chip communication infrastructure is a central component of modern systems-on-chip (SoCs), and it continues to gain importance as the number of cores, the heterogeneity of components, and the on-chip and off-chip bandwidth continue to grow. Decades of research on on-chip networks enabled cache-coherent shared-memory multiprocessors. However, communication fabrics that meet the needs of heterogeneous many-cores and accelerator-rich SoCs, which are not, or only partially, coherent, are a much less mature research area. In this work, we present a modular, topology-agnostic, high-performance on-chip communication platform. The platform includes components to build and link subnetworks with customizable bandwidth and concurrency properties and adheres to a state-of-the-art, industry-standard protocol. We discuss microarchitectural trade-offs and timing/area characteristics of our modules and show that they can be composed to build high-bandwidth (e.g., 2.5 GHz and 1024 bit data width) end-to-end on-chip communication fabrics (not only network switches but also DMA engines and memory controllers) with high degrees of concurrency. We design and implement a state-of-the-art ML training accelerator, where our communication fabric scales to 1024 cores on a die, providing 32 TB/s cross-sectional bandwidth at only 24 ns round-trip latency between any two cores.

show abstract

An Overview of Efficient Interconnection Networks for Deep Neural Network Accelerators

Cited by 54 publications

References 127 publications

Coordinated Batching and DVFS for DNN Inference on GPU Accelerators

Coordinated Batching and DVFS for DNN Inference on GPU Accelerators

A Study on the Relationship between Painter’s Psychology and Anime Creation Style Based on a Deep Neural Network

An Open-Source Platform for High-Performance Non-Coherent On-Chip Communication

Contact Info

Product

Resources

About