Impact of On-chip Interconnect on In-memory Acceleration of Deep Neural Networks

Abstract

With the widespread use of Deep Neural Networks (DNNs), machine learning algorithms have evolved in two diverse directions—one with ever-increasing connection density for better accuracy and the other with more compact sizing for energy efficiency. The increase in connection density increases on-chip data movement, which makes efficient on-chip communication a critical function of the DNN accelerator. The contribution of this work is threefold. First, we illustrate that the point-to-point (P2P)-based interconnect is incapable of handling a high volume of on-chip data movement for DNNs. Second, we evaluate P2P and network-on-chip (NoC) interconnect (with a regular topology such as a mesh) for SRAM- and ReRAM-based in-memory computing (IMC) architectures for a range of DNNs. This analysis shows the necessity for the optimal interconnect choice for an IMC DNN accelerator. Finally, we perform an experimental evaluation for different DNNs to empirically obtain the performance of the IMC architecture with both NoC-tree and NoC-mesh. We conclude that, at the tile level, NoC-tree is appropriate for compact DNNs employed at the edge, and NoC-mesh is necessary to accelerate DNNs with high connection density. Furthermore, we propose a technique to determine the optimal choice of interconnect for any given DNN. In this technique, we use analytical models of NoC to evaluate end-to-end communication latency of any given DNN. We demonstrate that the interconnect optimization in the IMC architecture results in up to 6 × improvement in energy-delay-area product for VGG-19 inference compared to the state-of-the-art ReRAM-based IMC architectures.

End-to-End Benchmarking of Chiplet-Based In-Memory Computing

“…The interface between NeuroSim and popular ML frameworks such as PyTorch and TensorFlow has also been created to make it more user-friendly [95]. However, one major drawback of NeuroSim is that it assumes H-Tree based bus interconnect for inter-tile communication, which can consume up to 90% of the total energy consumption of DNN inference [96]. To overcome this issue, Krishnan et al…”

Review of security techniques for memristor computing systems

“…Large NN layers are assigned to multiple PEs, and the partial sums are aggregated in a global buffer (Long et al, 2019;Krishnan et al, 2021). To maximize the processing parallelism of memristor crossbars, some memristor computing systems (Zhu et al, 2020;Wan et al, 2022) proposed to directly transfer the partial outputs of a NN layer to the PEs where its next layer is located.…”

“…The memory accessing patterns leak the NN structure information. For some memristor computing systems, a similar layer-by-layer processing technique is used (Qiao et al, 2018;Krishnan et al, 2021). Thus, the memory accessing patterns could also be a side-channel vulnerability that adversaries can exploit in memristor computing systems.…”

COIN: Communication-Aware In-Memory Acceleration for Graph Convolutional Networks

“…Thus, RRAM and SRAM-based IMC accelerators have been proposed for DNNs in the literature [19,25]. However, IMC increases on-chip data volume, which increases latency and energy due to on-chip communication [26][27][28][29]. The high density and complexity of GCNs make the on-chip communication for IMC-based accelerators even more critical.…”