Hierarchical and Parallel Pipelined Heterogeneous SoC for Embedded Vision Processing

Bin, Zhang; Zhao, Chen; Mei, Kuizhi; Zhao, Jizhong; Zheng, Nanning

doi:10.1109/tcsvt.2017.2665489

Cited by 8 publications

(2 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although recently hardware manufacturers launched new heterogeneous products, e.g., Xilinx Zynq Ultrascale+ MPSoC 1 (CPU, GPU and FPGA) and Altera SoC products 2 (CPU and FPGA), these are not fully exploited in computer vision domain (except handful of recent work, e.g., Zhang et al [73]) as majority of the existing algorithms are not designed to target heterogeneous platforms. Consideration of target hardware during the algorithmic development cycle is not always necessary and the domain experts often prototype new algorithms using library-rich languages such as MATLAB.…”

Section: Heterogeneous Computing For Vision Systemsmentioning

confidence: 99%

Embedded Vision Systems: A Review of the Literature

Bhowmik

Appiah

2018

Applied Reconfigurable Computing. Architectures, Tools, and Applications

View full text Add to dashboard Cite

Over the past two decades, the use of low power Field Programmable Gate Arrays (FPGA) for the acceleration of various vision systems mainly on embedded devices have become widespread. The reconfigurable and parallel nature of the FPGA opens up new opportunities to speed-up computationally intensive vision and neural algorithms on embedded and portable devices. This paper presents a comprehensive review of embedded vision algorithms and applications over the past decade. The review will discuss vision based systems and approaches, and how they have been implemented on embedded devices. Topics covered include image acquisition, preprocessing, object detection and tracking, recognition as well as high-level classification. This is followed by an outline of the advantages and disadvantages of the various embedded implementations. Finally, an overview of the challenges in the field and future research trends are presented. This review is expected to serve as a tutorial and reference source for embedded computer vision systems.

show abstract

Section: Heterogeneous Computing For Vision Systemsmentioning

confidence: 99%

Embedded Vision Systems: A Review of the Literature

Bhowmik

Appiah

2018

Applied Reconfigurable Computing. Architectures, Tools, and Applications

View full text Add to dashboard Cite

show abstract

“…For instance, the convolutional neural network vision recognition tasks [15][16][17][18], and all the conve on them. Therefore, many new specific designs have b processing in the last few years, called neural netw ditionally, a lot of effort has been made to shift th work [20][21][22][23][24] to reduce the expensive pixels transm be connected to the sensors directly, because they vided by the sensors. Some essential ISP tasks, suc on the RAW images first to convert them into the NPUs process them.…”

Section: Introductionmentioning

confidence: 99%

A Hybrid Vision Processing Unit with a Pipelined Workflow for Convolutional Neural Network Accelerating and Image Signal Processing

Liu

Yan

2021

Electronics

View full text Add to dashboard Cite

Vision processing chips have been widely used in image processing and recognition tasks. They are conventionally designed based on the image signal processing (ISP) units directly connected with the sensors. In recent years, convolutional neural networks (CNNs) have become the dominant tools for many state-of-the-art vision processing tasks. However, CNNs cannot be processed by a conventional vision processing unit (VPU) with a high speed. On the other side, the CNN processing units cannot process the RAW images from the sensors directly and an ISP unit is required. This makes a vision system inefficient with a lot of data transmission and redundant hardware resources. Additionally, many CNN processing units suffer from a low flexibility for various CNN operations. To solve this problem, this paper proposed an efficient vision processing unit based on a hybrid processing elements array for both CNN accelerating and ISP. Resources are highly shared in this VPU, and a pipelined workflow is introduced to accelerate the vision tasks. We implement the proposed VPU on the Field-Programmable Gate Array (FPGA) platform and various vision tasks are tested on it. The results show that this VPU achieves a high efficiency for both CNN processing and ISP and shows a significant reduction in energy consumption for vision tasks consisting of CNNs and ISP. For various CNN tasks, it maintains an average multiply accumulator utilization of over 94% and achieves a performance of 163.2 GOPS with a frequency of 200 MHz.

show abstract