Multimode fusion perception for transparent glass recognition

Abstract

Purpose The purpose of this paper is to present a novel tactile sensor and a visual-tactile recognition framework to reduce the uncertainty of the visual recognition of transparent objects. Design/methodology/approach A multitask learning model is used to recognize intuitive appearance attributes except texture in the visual mode. Tactile mode adopts a novel vision-based tactile sensor via the level-regional feature extraction network (LRFE-Net) recognition framework to acquire high-resolution texture information and temperature information. Finally, the attribute results of the two modes are integrated based on integration rules. Findings The recognition accuracy of attributes, such as style, handle, transparency and temperature, is near 100%, and the texture recognition accuracy is 98.75%. The experimental results demonstrate that the proposed framework with a vision-based tactile sensor can improve attribute recognition. Originality/value Transparency and visual differences make the texture of transparent glass hard to recognize. Vision-based tactile sensors can improve the texture recognition effect and acquire additional attributes. Integrating visual and tactile information is beneficial to acquiring complete attribute features.

Dance somersault gesture recognition based on multi-scale depth feature fusion

“…Early human behavior recognition relies mainly on local characteristics. Generally, the recognition process is as follows: firstly, human region detection is carried out, interest points are found from each frame of image, and then features are extracted from the motion changes of interest points [6]. Typical methods include Harris corner detection and dense trajectory.…”

Guest editorial: Dexterous manipulation

“…The third paper (Zhang et al , 2022) presents a vision-based tactile sensor and a visual–tactile recognition framework to reduce the uncertainty of the visual recognition of transparent objects. The tactile sensor was embedded into the palm of a dexterous hand.…”