Hybrid FPGA–CPU-Based Architecture for Object Recognition in Visual Servoing of Arm Prosthesis

Abstract

The present paper proposes an implementation of a hybrid hardware–software system for the visual servoing of prosthetic arms. We focus on the most critical vision analysis part of the system. The prosthetic system comprises a glass-worn eye tracker and a video camera, and the task is to recognize the object to grasp. The lightweight architecture for gaze-driven object recognition has to be implemented as a wearable device with low power consumption (less than 5.6 W). The algorithmic chain comprises gaze fixations estimation and filtering, generation of candidates, and recognition, with two backbone convolutional neural networks (CNN). The time-consuming parts of the system, such as SIFT (Scale Invariant Feature Transform) detector and the backbone CNN feature extractor, are implemented in FPGA, and a new reduction layer is introduced in the object-recognition CNN to reduce the computational burden. The proposed implementation is compatible with the real-time control of the prosthetic arm.

Application of Improved KAZE Algorithm in Image Feature Extraction and Matching

“…5: This method can effectively eliminate the influence of noise factors and maintain stable feature sizes. The grayscale information algorithm uses the comparison of image grayscale values on pixels to achieve matching, but this type of algorithm is highly susceptible to noise and image scale interference in information detection [14]. Algorithms such as SURF and SIFT use linear scale space and Gaussian filter technology, which will reduce the original features of the image to some extent.…”

“…Algorithms such as SURF and SIFT use linear scale space and Gaussian filter technology, which will reduce the original features of the image to some extent. The nonlinear scale space constructed by the KAZE algorithm can achieve high noise resistance and deformation resistance, and can better cope with issues such as changes in lighting intensity and angle size [14]. Thus, the research ultimately selected the KAZE algorithm for feature extraction, and provided a detailed description of the algorithm and its improved algorithms.…”