Efficient Deconvolution Architecture for Heterogeneous Systems-on-Chip

Abstract

Today, convolutional and deconvolutional neural network models are exceptionally popular thanks to the impressive accuracies they have been proven in several computer-vision applications. To speed up the overall tasks of these neural networks, purpose-designed accelerators are highly desirable. Unfortunately, the high computational complexity and the huge memory demand make the design of efficient hardware architectures, as well as their deployment in resource- and power-constrained embedded systems, still quite challenging. This paper presents a novel purpose-designed hardware accelerator to perform 2D deconvolutions. The proposed structure applies a hardware-oriented computational approach that overcomes the issues of traditional deconvolution methods, and it is suitable for being implemented within any virtually system-on-chip based on field-programmable gate array devices. In fact, the novel accelerator is simply scalable to comply with resources available within both high- and low-end devices by adequately scaling the adopted parallelism. As an example, when exploited to accelerate the Deep Convolutional Generative Adversarial Network model, the novel accelerator, running as a standalone unit implemented within the Xilinx Zynq XC7Z020 System-on-Chip (SoC) device, performs up to 72 GOPs. Moreover, it dissipates less than 500mW@200MHz and occupies 5.6%, 4.1%, 17%, and 96%, respectively, of the look-up tables, flip-flops, random access memory, and digital signal processors available on-chip. When accommodated within the same device, the whole embedded system equipped with the novel accelerator performs up to 54 GOPs and dissipates less than 1.8W@150MHz. Thanks to the increased parallelism exploitable, more than 900 GOPs can be executed when the high-end Virtex-7 XC7VX690T device is used as the implementation platform. Moreover, in comparison with state-of-the-art competitors implemented within the Zynq XC7Z045 device, the system proposed here reaches a computational capability up to 20% higher, and saves more than 60% and 80% of power consumption and logic resources requirement, respectively, using 5.7× fewer on-chip memory resources.

Design of Flexible Hardware Accelerators for Image Convolutions and Transposed Convolutions

“…Unfortunately, these characteristics may represent a bottleneck for those application scenarios in which real time and low power are mandatory. For this reason, designing ad-hoc hardware accelerators suitable for exploitation also within time- and power-constrained operating environments has recently received a great deal of attention [ 11 , 12 , 13 , 14 , 15 , 16 , 17 , 19 , 20 , 21 , 22 , 23 ]. Among the possible hardware realization platforms, FPGAs are widely recognized as powerful solutions [ 11 , 13 , 15 , 17 , 20 ] for merging the benefits from custom hardware designs, such as computational parallelism and limited energy consumption, with the strengths of software designs, including reconfigurability and short time to market.…”

“…While several of the existing hardware designs support both CONVs and TCONVs [ 11 , 13 , 14 , 15 , 16 , 17 , 19 , 21 ], some of them are tailored to accomplish only TCONVs [ 12 , 22 , 23 ]. As an example, the FPGA accelerator proposed in our previous work [ 12 ] deals with the input-oriented method (IOM) to reduce, or completely avoid, useless operations, corresponding to multiplications by zero, introduced by the conventional zero-TCONVs’ up-sampling approach. This is made possible by computing the products between each input pixel and the k × k elements of the filter, and then properly arranging the k × k results within the ofmap .…”

“…This is made possible by computing the products between each input pixel and the k × k elements of the filter, and then properly arranging the k × k results within the ofmap . Obviously, as a drawback, designs [ 12 , 22 , 23 ] need either additional buffers or auxiliary computing resources, or both, to manage row/column overlaps. Moreover, they may result quite inefficient when the CNN model being accelerated also uses CONV layers, as happens in the case of SR imaging applications [ 10 , 11 ].…”

“…The novel algorithm here presented exploits a computational strategy quite different than previous works [ 11 , 12 , 13 , 14 , 15 ]. In contrast to [ 11 , 13 , 14 , 15 ], which manipulate the k × k filter coefficients to form smaller sub-blocks (thus introducing the necessity of offline elaborations), and with respect to [ 12 ] that re-arranges the position of output values within the ofmaps (leading to area and time overhead due to the management of the overlapping regions), it applies an unconventional remapping strategy directly to the incoming ifmaps values. From a hardware perspective, this means that: (1) The process occurs online and the preprocessing is not required, and (2) the result of the proposed algorithm can be outputted as soon as it is produced, thus avoiding additional time and buffering/computing resources.…”

“…Indeed, as a distinctive feature, in order to up-sample low-resolution images, CNN-based SR algorithms typically adopt transposed convolutional (TCONV) layers [ 10 ] that, with their computational complexity up to 6.75 times higher than traditional convolutional (CONV) layers, represent the most critical component of CNNs [ 11 ]. Moreover, in comparison to CONVs, TCONV layers require more complex strategies to access data memory, and make skipping operations necessary to manage the incoming pixels properly [ 12 ]. In order to overcome the aforementioned issues, several algorithms have been proposed [ 11 , 13 , 14 , 15 ] to transform TCONV into CONV layers by pre-processing either the input data or the filter coefficients.…”