Hardware accelerator for high accuracy sign language recognition with residual network based on FPGAs

Abstract

The ResNet series of networks has demonstrated powerful capabilities in the fields of object detection and image classification, garnering increasing attention from researchers. However, due to their deep network architectures, accelerator development based on FPGA faces challenges associated with limited on-chip resources and lengthy design cycles. This paper presents a versatile hardware acceleration system based on FPGA, achieving optimization through both hardware implementation and software inference architecture. The system reduces network complexity by employing techniques such as inter-layer fusion and dynamic quantization, while enhancing hardware resource utilization through channel parallelism and tightly-pipelined hardware design principles. By configuring and reusing computation units, the forward inference process of ResNet series networks can be rapidly deployed on FPGA, shortening the development and validation cycles. The proposed system is validated using the ResNet-18 model on a PYNQ-Z2 development board within a gesture recognition application scenario. The overall power consumption of the system is 2.136W, with hardware inference accuracy reaching 98.87%.