Memory-Efficient Architecture for Accelerating Generative Networks on FPGA

Liu, Shuanglong; Zeng, Chenglong; Fan, Hongxiang; Ng, Ho-Cheung; Meng, Jiuxi; Que, Zhiqiang; Niu, Xinyu; Luk, Wayne

doi:10.1109/fpt.2018.00016

Cited by 20 publications

(17 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The obtained results are summarized in Table 1 in terms of: supported parallelism (T M , T N , P M , and P N ), kernel size (K) and stride (S); resources requirements; running frequency; number of operations performed per second (GOPs); and, finally, dynamic power consumption. It is worth highlighting that while the designs presented in [15][16][17] are SUs, those demonstrated in [9,11,18] are embedded heterogeneous systems (ESs). For this reason, several SU and ES versions of the design here presented have been characterized and they are referenced in Table 1.…”

Section: Implementation and Resultsmentioning

confidence: 99%

Efficient Deconvolution Architecture for Heterogeneous Systems-on-Chip

et al. 2020

View full text Add to dashboard Cite

Today, convolutional and deconvolutional neural network models are exceptionally popular thanks to the impressive accuracies they have been proven in several computer-vision applications. To speed up the overall tasks of these neural networks, purpose-designed accelerators are highly desirable. Unfortunately, the high computational complexity and the huge memory demand make the design of efficient hardware architectures, as well as their deployment in resource- and power-constrained embedded systems, still quite challenging. This paper presents a novel purpose-designed hardware accelerator to perform 2D deconvolutions. The proposed structure applies a hardware-oriented computational approach that overcomes the issues of traditional deconvolution methods, and it is suitable for being implemented within any virtually system-on-chip based on field-programmable gate array devices. In fact, the novel accelerator is simply scalable to comply with resources available within both high- and low-end devices by adequately scaling the adopted parallelism. As an example, when exploited to accelerate the Deep Convolutional Generative Adversarial Network model, the novel accelerator, running as a standalone unit implemented within the Xilinx Zynq XC7Z020 System-on-Chip (SoC) device, performs up to 72 GOPs. Moreover, it dissipates less than 500mW@200MHz and occupies 5.6%, 4.1%, 17%, and 96%, respectively, of the look-up tables, flip-flops, random access memory, and digital signal processors available on-chip. When accommodated within the same device, the whole embedded system equipped with the novel accelerator performs up to 54 GOPs and dissipates less than 1.8W@150MHz. Thanks to the increased parallelism exploitable, more than 900 GOPs can be executed when the high-end Virtex-7 XC7VX690T device is used as the implementation platform. Moreover, in comparison with state-of-the-art competitors implemented within the Zynq XC7Z045 device, the system proposed here reaches a computational capability up to 20% higher, and saves more than 60% and 80% of power consumption and logic resources requirement, respectively, using 5.7× fewer on-chip memory resources.

show abstract

Section: Implementation and Resultsmentioning

confidence: 99%

Efficient Deconvolution Architecture for Heterogeneous Systems-on-Chip

et al. 2020

View full text Add to dashboard Cite

show abstract

“…The distributions P A and P B are the probability of transition between A and B states are shown in Eqs. (3) and (4), where β = − V T 1 is a thermal parameter, and α = Δt∕t c . The parameter Δt is time step ratio, and t c is a time scale of the device 32 .…”

Section: Methods Technology Models and Tools The System Level Simulmentioning

confidence: 99%

Memristive GAN in Analog

Krestinskaya¹,

Choubey

James

2020

Sci Rep

View full text Add to dashboard Cite

Generative Adversarial network (GAn) requires extensive computing resources making its implementation in edge devices with conventional microprocessor hardware a slow and difficult, if not impossible task. in this paper, we propose to accelerate these intensive neural computations using memristive neural networks in analog domain. the implementation of Analog Memristive Deep convolutional GAn (AM-DcGAn) using Generator as deconvolutional and Discriminator as convolutional memristive neural network is presented. The system is simulated at circuit level with 1.7 million memristor devices taking into account memristor non-idealities, device and circuit parameters. the design is modular with crossbar arrays having a minimum average power consumption per neural computation of 47nW. The design exclusively uses the principles of neural network dropouts resulting in regularization and lowering the power consumption. the Spice level simulation of GAn is performed with 0.18 μm CMOS technology and WO x memristive devices with R ON = 40 kΩ and R OFF = 250 kΩ, threshold voltage 0.8 V and write voltage at 1.0 V. Intelligent near-sensor analog computing requires integration of neural co-processor unit next to the sensing unit. The analog memristive neuromorphic circuits 1 can integrate complex neural networks and learning systems to edge devices and sensors due to the advantages offered by memristors in area efficiency, non-volatility, programmability, high switching speeds, and low power requirements. Generative Adversarial Network (GAN) is a well known computationally complex neural network architecture for generating new patterns that requires significant computational resources in software implementations as well as large amount of data for training 2. This makes its implementation in edge devices with conventional microprocessor hardware a slow and difficult task. Recently, there have been several works proposing to accelerate GAN and implement the generative networks on FPGA 3,4 and digital CMOS circuits 5. These solutions have high power consumption and on-chip area for implementing within a edge device. The implementation of GAN on edge devices will eliminate the requirement of sending large amount of data collected by a sensor (e.g. in a camera) to a server for processing, which can speed up the GAN training on low power devices, where on-chip area, memory and power consumption is limited. The application of memristive devices for GAN can be a promising solution for near-sensor edge computing due to small on-chip area and low power consumption. Previously, memristive devices have been used for accelerating the vector-matrix multiplication operations in the neural networks and learning systems on hardware 6. Recently, a 128 × 64 reconfigurable hafnium oxide memristive crossbar has been tested for analogue vector-matrix multiplication with high device yield and high state retention for image processing applications 7. An integrated 128 × 64 1T1R crossbar array has been used in a Long-Short Term Memory (LSTM) network with...

show abstract

“…Few works have focused on accelerating DeConv layers and generative networks (GANs) [11], [18], [19]. Yazdanbakhsh et al [18] proposed an end-to-end FPGA accelerator for GANs that combined MIMD and SIMD models while separating data retrieval and data processing units at the finest granularity.…”

Section: B Related Workmentioning

confidence: 99%

“…Yan et al [19] proposed a dual convolution mapping method to make full use of the available computational resources. Liu et al [11] proposed a novel layer fusion method for GANs with a memory-efficient architecture to significantly reduce off-chip data transfers. However, these researchers addressed the accelerations of deconvolution in generative adversarial networks and didn't aim at segmentation problems.…”

Section: B Related Workmentioning

confidence: 99%

“…FPGAs have shown very good performance for DNN-based segmentation acceleration [6], [11], and thus are considered promising to accelerate RSI segmentation methods in space platforms. The main challenge and limitations of previous designs of segmentation on FPGA are that: 1) unlike classification or detection networks such as VGG- 16 or AlexNet, segmentation networks consist of both convolutional (Conv) and deconvolutional (Deconv) layers which require different mathematical operations, making it difficult to design efficient architectures for them; 2) Deconv must insert zeros into the input image when implemented as Conv, leading to the sparsity of input as well as computational inefficiency due to multiplications with zeros [11].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Towards an Efficient Accelerator for DNN-Based Remote Sensing Image Segmentation on FPGAs

Liu

Luk

2019

2019 29th International Conference on Field Programmable Logic and Applications (FPL)

Self Cite

View full text Add to dashboard Cite

Among popular techniques in remote sensing image (RSI) segmentation, Deep Neural Networks (DNNs) have gained increasing interest but often require high computation complexity, which largely limit their applicability in on-board space platforms. Therefore, various dedicated hardware designs on FPGAs have been developed to accelerate DNNs. However, it imposes difficulty on the design of efficient accelerator for DNN-based segmentation algorithms, since they need to perform both convolution and deconvolution which are two fundamentally different types of operations. This paper proposes a uniform architecture to efficiently implement both convolution and deconvolution in one vector multiplication module. This architecture is further optimized through exploiting different levels of parallelism and layer fusion to achieve low latency for RSI segmentation tasks. Moreover, an optimized DNN model is developed for real-time RSI segmentation, which shows preferable accuracy compared to other methods. The proposed hardware accelerator efficiently implements the DNN model on Intel's Arria 10 device, demonstrating 1578 GOPS of throughput and 17.4 ms of latency, i.e., 57 images per second.

show abstract

Memory-Efficient Architecture for Accelerating Generative Networks on FPGA

Cited by 20 publications

References 17 publications

Efficient Deconvolution Architecture for Heterogeneous Systems-on-Chip

Efficient Deconvolution Architecture for Heterogeneous Systems-on-Chip

Memristive GAN in Analog

Towards an Efficient Accelerator for DNN-Based Remote Sensing Image Segmentation on FPGAs

Contact Info

Product

Resources

About