Automatic Deployment of Convolutional Neural Networks on FPGA for Spaceborne Remote Sensing Application

Abstract

In recent years, convolutional neural network (CNN)-based algorithms have been widely used in remote sensing image processing and show tremendous performance in a variety of application fields. However, large amounts of data and intensive computations make the deployment of CNN-based algorithms a challenging problem, especially for the spaceborne scenario where resources and power consumption are limited. To tackle this problem, this paper proposes an automatic CNN deployment solution on resource-limited field-programmable gate arrays (FPGAs) for spaceborne remote sensing applications. Firstly, a series of hardware-oriented optimization methods are proposed to reduce the complexity of the CNNs. Secondly, a hardware accelerator is designed. In this accelerator, a reconfigurable processing engine array with efficient convolutional computation architecture is used to accelerate CNN-based algorithms. Thirdly, to bridge the optimized CNNs and hardware accelerator, a compilation toolchain is introduced into the deployment solution. Through the automatic conversion from CNN models to hardware instructions, various networks can be deployed on hardware in real-time. Finally, we deployed an improved VGG16 network and an improved YOLOv2 network on Xilinx AC701 to evaluate the effectiveness of the proposed deployment solution. The experiments show that with only 3.407 W power consumption and 94 DSP consumption, our solution achieves 23.06 giga operations per second (GOPS) throughput in the improved VGG16 and 22.17 GOPS throughput in the improved YOLOv2. Compared to the related works, the DSP efficiency of our solution is improved by 1.3–2.7×.

A Lightweight Detection Method for Remote Sensing Images and Its Energy-Efficient Accelerator on Edge Devices

“…In addition, we compared the proposed CNN accelerator with previous works [ 54 , 55 , 56 , 57 , 58 , 59 ], as shown in Table 8 . Ref.…”

“…[ 59 ] proposed an automatic CNN deployment solution on resource-limited devices. Both works [ 58 , 59 ] used less DSP resources to implement the inference of the CNN. However, the accelerator in Refs.…”

“…However, the accelerator in Refs. [ 58 , 59 ] only achieves 10.25 GOPS and 22.17 GOPS throughput, respectively. Compared with the design where the whole network is repeatedly stacked with modules in the order of the respective layer, the proposed CNN accelerator can perform different types of convolution computing using a single PE array, such that the efficiency of hardware resources can be significantly increased.…”

Exploring the Advancements in High-Performance Computing Paradigm for Remote Sensing Big Data Analytics

“…A collaborative platform for offering data, algorithms, processing and analytic services to a number of users from different public and private user communities is provided [24]. A Convolutional Neural Network (CNN)based deployment solution on resource-limited FPGA for spaceborne applications using RSBD is implemented [25]. A scalable computing resource model is developed to achieve fast processing of RSBD using a parallel distributed architecture [26].…”

Reduced-Parameter YOLO-like Object Detector Oriented to Resource-Constrained Platform

“…For the accelerator, it is finally to be deployed in the actual application environment, which is an important research topic for hardware accelerators. Prior to this, there have been many studies on the practical application of neural network accelerators [ 12 , 13 , 14 , 15 ] and the design of neural network accelerators [ 16 , 17 , 18 ]. However, most of the existing algorithm models are transformed and arranged on the FPGA, and the model design is not combined with the hardware architecture.Therefore, in this work, we design a YOLO algorithm and deploy the algorithm model to run on FPGAs.…”