TOSQ: Transparent object segmentation through query based transformer

Abstract

Sensing transparent objects has many applications in human daily life, including robot navigation and grasping. However, this task presents significant challenges due to the unpredictable nature of scenes that lay beyond transparent objects. This paper aims to solve the transparent object segmentation problem based Transformer. We design a Query Parsing Module (QPM) that formulates the transparent object segmentation task into a dictionary look-up problem and a set of learnable class prototypes as query inputs. Based QPM, we propose a high-performance transformer-based end-to-end segmentation model Transparent Object Segmentation through Query (TOSQ). TOSQ’s encoder is based on the Segformer’s backbone, and its decoder consists of a series of QPM modules. On the Trans10K-V2 dataset, TOSQ significantly outperforms almost all CNN-based and transformer-based methods, fully demonstrating the unique advantages and great potential of TOSQ to solve the semantic segmentation problem of transparent objects in daily human life. The code is publicly available at https://github.com/ldepn/tosq.