PolarMask: Single Shot Instance Segmentation With Polar Representation

Xie, Enze; Sun, Peize; Song, Xiaoge; Wang, Wenhai; Liu, Xuebo; Ding, Liang; Shen, Chunhua; Luo, Ping

doi:10.1109/cvpr42600.2020.01221

Cited by 513 publications

(316 citation statements)

References 24 publications

Supporting

Mentioning

313

Contrasting

Unclassified

Order By: Relevance

“…The modern development of two-stage segmentation is the relatively fast model YOLACT++ [21]. Another approach to improving the quality of segmentation of found objects involves deformation of the found contour with a special neural network, for example, based on the polar representation of the contour in PolarMask [22], the concept of the circular convolution in Deep Snake [23], or deep polygon transformer in PolyTransorfm [24].…”

Section: B Real-time Object Segmentationmentioning

confidence: 99%

Real-Time Object Navigation With Deep Neural Networks and Hierarchical Reinforcement Learning

et al. 2020

View full text Add to dashboard Cite

In the last years, deep learning and reinforcement learning methods have significantly improved mobile robots in such fields as perception, navigation, and planning. But there are still gaps in applying these methods to real robots due to the low computational efficiency of recent neural network architectures and their poor adaptability to robotic experiments' realities. In this paper, we consider an important task in mobile robotics -navigation to an object using an RGB-D camera. We develop a new neural network framework for robot control that is fast and resistant to possible noise in sensors and actuators. We propose an original integration of semantic segmentation, mapping, localization, and reinforcement learning methods to improve the effectiveness of exploring the environment, finding the desired object, and quickly navigating to it. We created a new HISNav dataset based on the Habitat virtual environment, which allowed us to use simulation experiments to pre-train the model and then upload it to a real robot. Our architecture is adapted to work in a real-time environment and fully implements modern trends in this area.

show abstract

Section: B Real-time Object Segmentationmentioning

confidence: 99%

Real-Time Object Navigation With Deep Neural Networks and Hierarchical Reinforcement Learning

et al. 2020

View full text Add to dashboard Cite

show abstract

“…With continual advancement in statistical modeling, speech recognition has been widely adopted in robots and smart devices (Reddy and Raj, 1976 ) to realize natural language-based human–computer interaction. Furthermore, substantial development in the field of image perception has been carried out, even achieving human-level performance in some tasks (Hou et al, 2020 ; Uzkent et al, 2020 ; Xie et al, 2020 ). By fusing visual and auditory information, robots are able to understand human natural language instructions and carry out required tasks.…”

Section: Introductionmentioning

confidence: 99%

Intention Understanding in Human–Robot Interaction Based on Visual-NLP Semantics

Sun

et al. 2021

Front. Neurorobot.

View full text Add to dashboard Cite

With the rapid development of robotic and AI technology in recent years, human–robot interaction has made great advancement, making practical social impact. Verbal commands are one of the most direct and frequently used means for human–robot interaction. Currently, such technology can enable robots to execute pre-defined tasks based on simple and direct and explicit language instructions, e.g., certain keywords must be used and detected. However, that is not the natural way for human to communicate. In this paper, we propose a novel task-based framework to enable the robot to comprehend human intentions using visual semantics information, such that the robot is able to satisfy human intentions based on natural language instructions (total three types, namely clear, vague, and feeling, are defined and tested). The proposed framework includes a language semantics module to extract the keywords despite the explicitly of the command instruction, a visual object recognition module to identify the objects in front of the robot, and a similarity computation algorithm to infer the intention based on the given task. The task is then translated into the commands for the robot accordingly. Experiments are performed and validated on a humanoid robot with a defined task: to pick the desired item out of multiple objects on the table, and hand over to one desired user out of multiple human participants. The results show that our algorithm can interact with different types of instructions, even with unseen sentence structures.

show abstract

“…Supervised deep learning enables accurate computer vision models. Key for this success is the access to raw sensor data (i.e., images) with ground truth (GT) for the visual task at hand (e.g., image classification [ 1 ], object detection [ 2 ] and recognition [ 3 ], pixel-wise instance/semantic segmentation [ 4 , 5 ], monocular depth estimation [ 6 ], 3D reconstruction [ 7 ], etc.). The supervised training of such computer vision models, which are based on convolutional neural networks (CNNs), is known to required very large amounts of images with GT [ 8 ].…”

Section: Introductionmentioning

confidence: 99%

Co-Training for Deep Object Detection: Comparing Single-Modal and Multi-Modal Approaches

Gómez

Villalonga

López

2021

Sensors

View full text Add to dashboard Cite

Top-performing computer vision models are powered by convolutional neural networks (CNNs). Training an accurate CNN highly depends on both the raw sensor data and their associated ground truth (GT). Collecting such GT is usually done through human labeling, which is time-consuming and does not scale as we wish. This data-labeling bottleneck may be intensified due to domain shifts among image sensors, which could force per-sensor data labeling. In this paper, we focus on the use of co-training, a semi-supervised learning (SSL) method, for obtaining self-labeled object bounding boxes (BBs), i.e., the GT to train deep object detectors. In particular, we assess the goodness of multi-modal co-training by relying on two different views of an image, namely, appearance (RGB) and estimated depth (D). Moreover, we compare appearance-based single-modal co-training with multi-modal. Our results suggest that in a standard SSL setting (no domain shift, a few human-labeled data) and under virtual-to-real domain shift (many virtual-world labeled data, no human-labeled data) multi-modal co-training outperforms single-modal. In the latter case, by performing GAN-based domain translation both co-training modalities are on par, at least when using an off-the-shelf depth estimation model not specifically trained on the translated images.

show abstract

PolarMask: Single Shot Instance Segmentation With Polar Representation

Cited by 513 publications

References 24 publications

Real-Time Object Navigation With Deep Neural Networks and Hierarchical Reinforcement Learning

Real-Time Object Navigation With Deep Neural Networks and Hierarchical Reinforcement Learning

Intention Understanding in Human–Robot Interaction Based on Visual-NLP Semantics

Co-Training for Deep Object Detection: Comparing Single-Modal and Multi-Modal Approaches

Contact Info

Product

Resources

About