A CNN-Based Multi-scale Super-Resolution Architecture on FPGA for 4K/8K UHD Applications

Kim, Yongwoo; Choi, Jae-Seok; Lee, Jaehyup; Kim, Munchurl

doi:10.1007/978-3-030-37734-2_63

Cited by 3 publications

(2 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Kim et al [10] introduced an efficient multiscale SR hardware based on CNN that can convert 4 K UHD amplifiers to 60 fps. The multi-scale SR tool uses a limited number of filter parameters and does not use a frame buffer to store intermediate feature maps.…”

Section: Related Workmentioning

confidence: 99%

FPGA Implementation of Deep Leaning Model for Video Analytics

Khayam¹,

Mehmood²,

Chaudhry³

et al. 2022

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

In recent years, deep neural networks have become a fascinating and influential research subject, and they play a critical role in video processing and analytics. Since, video analytics are predominantly hardware centric, exploration of implementing the deep neural networks in the hardware needs its brighter light of research. However, the computational complexity and resource constraints of deep neural networks are increasing exponentially by time. Convolutional neural networks are one of the most popular deep learning architecture especially for image classification and video analytics. But these algorithms need an efficient implement strategy for incorporating more real time computations in terms of handling the videos in the hardware. Field programmable Gate arrays (FPGA) is thought to be more advantageous in implementing the convolutional neural networks when compared to Graphics Processing Unit (GPU) in terms of energy efficient and low computational complexity. But still, an intelligent architecture is required for implementing the CNN in FPGA for processing the videos. This paper introduces a modern high-performance, energy-efficient Bat Pruned Ensembled Convolutional networks (BPEC-CNN) for processing the video in the hardware. The system integrates the Bat Evolutionary Pruned layers for CNN and implements the new shared Distributed Filtering Structures (DFS) for handing the filter layers in CNN with pipelined data-path in FPGA. In addition, the proposed system adopts the hardware-software co-design methodology for an energy efficiency and less computational complexity. The extensive experimentations are carried out using CASIA video datasets with ARTIX-7 FPGA boards (number) and various algorithms centric parameters such as accuracy, sensitivity, specificity and architecture centric parameters such as the power, area and throughput are analyzed. These results are then compared with the existing pruned CNN architectures such as CNN-Prunner in which the proposed architecture has been shown 25% better performance than the existing architectures.

show abstract

Section: Related Workmentioning

confidence: 99%

FPGA Implementation of Deep Leaning Model for Video Analytics

Khayam¹,

Mehmood²,

Chaudhry³

et al. 2022

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

show abstract

“…SR technologies have been covering many areas such as medical imaging [16], surveillance [17], and depth maps [18]. Recently, as the resolutions of televisions are increasing to 8K (7680×4320), SR methods are getting more interests [19]. The SR is an ill-posed problem because there are always multiple HR images corresponding to a given LR image [20].…”

Section: Introductionmentioning

confidence: 99%

Complexity-Reduced Super Resolution for Foveation-Based Driving Head Mounted Displays

Nam

Kang

2021

IEEE Access

View full text Add to dashboard Cite

In this paper, we propose a foveation-based super resolution (SR) algorithm to create high resolution images from low resolution inputs for virtual reality head mounted displays. Because the proposed SR algorithm is integrated in the previous foveation-based driving technology to cover the small area around the foveation point that requires high rendering quality, the overall computational complexity is substantially reduced, compared to the whole area SR. The target display has 4 times as high resolution as the input image, therefore, the proposed SR algorithm generates 4× as well as 2× SR images at the same time. To support two SR output images, small area SR, and small number of weights, we employ cropping as well as progressive and recursive framework used in the previous MS-LapSRN. We reduce the number of neurons by placing the deconvolutional layer after convolutional layers, compared to the MS-LapSRN. PSNR and SSIM performances of the proposed SR for the 4× scale are estimated as 31.152 dB and 0.935 for Set5, 26.656 dB and 0.858 for Set14, 27.138 dB and 0.830 for BSD100, and 25.078 dB and 0.836 for Urban100. For the target 8K display of 7,680×4,320, the proposed FovSR-integrated driving technology achieves the substantial reductions by 76.7 % and 99.02 % on the number of lines from 7,680 to 1,788 and the number of neurons from 24,518,246,400 to 239,541,184, respectively.

show abstract

ADAS: A High Computational Utilization D ynamic Reconfigurable Hardware A ccelerator for S uper Resolution

Chang

Zhao

Zhou

2023

ACM Trans. Reconfigurable Technol. Syst.

View full text Add to dashboard Cite

Super-resolution (SR) based on deep learning has obtained superior performance in image reconstruction. Recently, various algorithm efforts have been committed to improving image reconstruction quality and speed. However, the inference of SR contains huge amounts of computation and data access, leading to low hardware implementation efficiency. For instance, the up-sampling with the deconvolution process requires considerable computation resources. In addition, the sizes of output feature maps of several middle layers are extraordinarily large, which is challenging to optimize, causing serious data access issues. In this work, we present an all-on-chip hardware architecture based on the deconvolution scheme and feature map segmentation strategy, namely ADAS, where all the generated data by the middle layers are buffered on-chip to avoid large data movements between on- and off-chip. In ADAS, we develop a hardware-friendly and efficient deconvolution scheme to accelerate the computation. Also, the dynamic reconfigurable process element (PE) combined with efficient mapping is proposed to enhance PE utilization up to nearly \(100\% \) and support multiple scaling factors. Based on our experimental results, ADAS demonstrates real-time image SR and better image reconstruction quality with PSNR (37.15 dB ) and SSIM (0.9587). Compared to baseline and validated with the FPGA platform, ADAS can support scaling factors of 2, 3, and 4, achieving 2.68 ×, 5.02 × and 8.28 × speedup.

show abstract

A CNN-Based Multi-scale Super-Resolution Architecture on FPGA for 4K/8K UHD Applications

Cited by 3 publications

References 3 publications

FPGA Implementation of Deep Leaning Model for Video Analytics

FPGA Implementation of Deep Leaning Model for Video Analytics

Complexity-Reduced Super Resolution for Foveation-Based Driving Head Mounted Displays

ADAS: A High Computational Utilization D ynamic Reconfigurable Hardware A ccelerator for S uper Resolution

Contact Info

Product

Resources

About