“What’s This?” - Learning to Segment Unknown Objects from Manipulation Sequences

and safety-critical real world applications remains limited. The main factors responsible for this limitation are• the lack of expressiveness and transparency of a deep neural network's inference model, which makes it difficult to trust their outcomes [2],• the inability to distinguish between in-domain and out-ofdomain samples [11], [12] and the sensitivity to domain shifts [13],• the inability to provide reliable uncertainty estimates for a deep neural network's decision [14] and frequently occurring overconfident predictions [15], [16],• the sensitivity to adversarial attacks that make deep neural networks vulnerable for sabotage [17], [18], [19].

show abstract

“…active perception [316]) to learning (e.g. active learning [317], [318], [319] and identifying unknown objects [320], [321], [322]).…”

Section: A Uncertainty In Real-world Applicationsmentioning

confidence: 99%

A Survey of Uncertainty in Deep Neural Networks

Gawlikowski¹,

Tassi²,

Ali³

et al. 2021

Preprint

Self Cite

124

158

View full text Add to dashboard Cite

show abstract

“…Specifically, as Fig. 2 shows, we first obtain the segmentation maps of the hand and object from N seg , an off-the-shelf method (Boerdijk et al 2021). Then, the segmentation maps of hand and object M are concatenated and fed into N hg amodal to generate the amodal results.…”

Section: Object Reconstructionmentioning

confidence: 99%

In-Hand 3D Object Reconstruction from a Monocular RGB Video

Jiang,

Ye,

Xie

et al. 2024

AAAI

View full text Add to dashboard Cite

Our work aims to reconstruct a 3D object that is held and rotated by a hand in front of a static RGB camera. Previous methods that use implicit neural representations to recover the geometry of a generic hand-held object from multi-view images achieved compelling results in the visible part of the object. However, these methods falter in accurately capturing the shape within the hand-object contact region due to occlusion. In this paper, we propose a novel method that deals with surface reconstruction under occlusion by incorporating priors of 2D occlusion elucidation and physical contact constraints. For the former, we introduce an object amodal completion network to infer the 2D complete mask of objects under occlusion. To ensure the accuracy and view consistency of the predicted 2D amodal masks, we devise a joint optimization method for both amodal mask refinement and 3D reconstruction. For the latter, we impose penetration and attraction constraints on the local geometry in contact regions. We evaluate our approach on HO3D and HOD datasets and demonstrate that it outperforms the state-of-the-art methods in terms of reconstruction surface quality, with an improvement of 52% on HO3D and 20% on HOD. Project webpage: https://east-j.github.io/ihor.

show abstract

“…In addition to leveraging 3D models and templates, other approaches have also been explored in related studies. [36] proposed a novel framework for detecting the grasped object solely by observing the motion between consecutive RGB frames and distinguishing between a manipulator and an object. However, static scenarios are difficult to detect.…”

Section: Other Approachesmentioning

confidence: 99%

Template-Based Category-Agnostic Instance Detection for Robotic Manipulation

Tan

Zhou

et al. 2022

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

An intelligent robotic system is one of the key pillars of a smart factory that requires flexibility to handle a variety of tasks. Perception is a key enabling technology for robots. Most existing object detection studies have mainly focused on category-specific objects and have achieved impressive performance. However, robotic systems, particularly in industrial scenarios, typically interact with many category-agnostic objects, which the robot must detect instantly without pre-training. Therefore, in this study, we proposed a template-based detection and segmentation approach, which incorporated a multi-level correlation model and a similarity-refine module, for handling the category-agnostic instance. The proposed approach was then validated and demonstrated in an interactive and adaptive robotic application scenario designed for the typical pick-and-place task. Among them, the picking scan path and location were instructed through human guidance with hand tracking. The neural rendering technology was also introduced to render novel views of the template. The proposed approach was evaluated using a benchmark and verified through a real demonstration.

show abstract

“What’s This?” - Learning to Segment Unknown Objects from Manipulation Sequences

Cited by 5 publications

References 44 publications

A Survey of Uncertainty in Deep Neural Networks

A Survey of Uncertainty in Deep Neural Networks

In-Hand 3D Object Reconstruction from a Monocular RGB Video

Template-Based Category-Agnostic Instance Detection for Robotic Manipulation

Contact Info

Product

Resources

About