Dataflow-Architecture Co-Design for 2.5D DNN Accelerators using Wireless Network-on-Package

Abstract

Deep neural network (DNN) models continue to grow in size and complexity, demanding higher computational power to enable real-time inference. To efficiently deliver such computational demands, hardware accelerators are being developed and deployed across scales. This naturally requires an efficient scale-out mechanism for increasing compute density as required by the application. 2.5D integration over interposer has emerged as a promising solution, but as we show in this work, the limited interposer bandwidth and multiple hops in the Network-on-Package (NoP) can diminish the benefits of the approach. To cope with this challenge, we propose WIENNA, a wireless NoP-based 2.5D DNN accelerator. In WIENNA, the wireless NoP connects an array of DNN accelerator chiplets to the global buffer chiplet, providing highbandwidth multicasting capabilities. Here, we also identify the dataflow style that most efficienty exploits the wireless NoP's high-bandwidth multicasting capability on each layer. With modest area and power overheads, WIENNA achieves 2.2X-5.1X higher throughput and 38.2% lower energy than an interposer-based NoP design.

Wireless On-Chip Communications for Scalable In-memory Hyperdimensional Computing

“…WNoCs have been introduced, among other alternatives, to overcome these issues. WNoCs are the result of augmenting cores or groups of cores with RF transceivers and antennas allowing them to communicate wirelessly through the chip package with all cores that are within range [44]- [46]. Even though this technology is still under development, proof-of-concept designs have been successfully implemented and tested [47].…”

“…Among the key advantages of WNoCs, one can find a natural support to broadcast communications, reduced latency, and an adaptive network topology [36], [39], [48], [49]. Hence, WNoCs can be especially advantageous if they are used to serve specific communication patterns that are very challenging to tackle using conventional NoCs [46]. This is of relevance in this work, as HDC algorithms being executed over IMC platforms make an intensive use of broadcast and reduction patterns, leading to important bottlenecks when scaled over traditional NoC/NiP platforms.…”

“…In this case, however, the interposer becomes a bottleneck in terms of bandwidth and connectivity due to I/O pin limitations. This leads to multi-hop and serial-link schemes that add significant energy and latency per hop, i.e., ∼1 pJ and ∼20 ns [50], with hop counts typically scaling with √ N for unicasts and with N for broadcasts [46].…”

Scale up your In-Memory Accelerator: Leveraging Wireless-on-Chip Communication for AIMC-based CNN Inference

“…Researchers have investigated different solutions to meet the requirements of current complex networks [6]- [8], but without detailing how in such complex systems and architectures, interconnections can sustain the many-AIMC throughput [9]. Novel approaches tackle this problem by proposing innovative technologies that can complement the wired interconnects with on-chip wireless communication channels [10]- [12], exploiting the higher bandwidth, versatility, plasticity, and energy efficiency to scale up the number of AIMC devices that can be integrated with sufficient on-chip bandwidth.…”

“…Moreover, computing packages are compatible with multiple wideband channels in frequency bands beyond 60 GHz, leading to hypothetical bandwidths in the order of tens or hundreds of Gb/s [14]. Finally, such an interconnect provides system-level versatility as bandwidth can be shared dynamically among the antennas adapting to the architecture requirements [10].…”