Grasp-Oriented Fine-grained Cloth Segmentation without Real Supervision

Ren, Ruijie; Rajesh, Mohit Gurnani; Sanchez-Riera, Jordi; Zhang, Fan; Tian, Yanling; Agudo, Antonio; Demiris, Yiannis; Mikolajczyk, Krystian; Moreno-Noguer, Francesc

doi:10.48550/arxiv.2110.02903

Cited by 2 publications

(5 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Reducing the cost of data collection and labeling while enhancing the generalization of the model is a major problem in robotics-related research. Some studies use physics engines such as Maya or Blender to synthesize large amounts of data [1], [5], [6], [8], [9], [25], but the distribution of synthetic data and real data is vastly different. This leads to a requirement for domain adaptation processing [1], [8], [9].…”

Section: B Data Synthesis and Augmentationmentioning

confidence: 99%

“…Some studies use physics engines such as Maya or Blender to synthesize large amounts of data [1], [5], [6], [8], [9], [25], but the distribution of synthetic data and real data is vastly different. This leads to a requirement for domain adaptation processing [1], [8], [9]. However while this processing does reduce the domain gap, it limits the ability of the model to generalize and as a result limits the effectiveness of synthetic data for improving accuracy.…”

Section: B Data Synthesis and Augmentationmentioning

confidence: 99%

“…Current state-of-the-art research addresses this by identifying graspable regions, as opposed to keypoints, and selecting graspable points from within these regions. These methods often rely solely on depth information to perform segmentation, however this is often susceptible to noise in the depth data [1], [8], [33], [34]. Alternatively, RGB only segmentation methods struggle with single color distributions often present in clothing.…”

Section: Robotic Manipulation Of Flexible Objectsmentioning

confidence: 99%

“…Most existing works apply deep learning methods for keypoints recognition leveraging depth images of the clothing. Due to collection of real labeled depth images being extremely time consuming, physics engines such as Maya and Blender are usually used for data enrichment [1], [5], [6], [8], [9]. However, a clear domain gap is present between the synthesized and real depth images, with the real depth images containing significant noise.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Salient Semantic Segmentation Based on RGB-D Camera for Robot Semantic Mapping

et al. 2023

View full text Add to dashboard Cite

Semantic mapping can help robots better understand the environment and is extensively studied in robotics. However, it is a challenge for semantic mapping that calibrates all the obstacles with semantics. We propose integrating two network models to realize the salient semantic segmentation used for mobile robot mapping, which differs from traditional segmentation methods. Firstly, we detected salient objects. The detection result was the grayscale image form, which was recognized and annotated by our trained model. Then, we projected the salient objects’ contour with semantics to the corresponding RGB image, which realized the salient objects’ semantic segmentation. We treated the salient objects instead of all the obstacles as semantic segmentation objects that could reduce the background consideration. The neural network model trained based on the salient object’s shape information was stable for object recognition and easy for model training. We only used the shape feature for training, which could reduce the calculation amount of feature details. Experiments demonstrated that the algorithm could quickly realize the model’s training and provide a semantic landmark in the point cloud map as the relative position reference for robot repositioning when the map needs to be used again and exist in a similar environment.

show abstract

Section: B Data Synthesis and Augmentationmentioning

confidence: 99%

Section: B Data Synthesis and Augmentationmentioning

confidence: 99%

Section: Robotic Manipulation Of Flexible Objectsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Salient Semantic Segmentation Based on RGB-D Camera for Robot Semantic Mapping

et al. 2023

View full text Add to dashboard Cite

show abstract

“…To avoid multiple grasping strategies, active search with the use of different neural networks has been employed to recognize two grasping points [5]. Finally, semantic area segmentation and domain adaptation were used to identify grasping points from a single shot image with the help of synthetic data [15]. Significant progress has been made on tasks such as folding, unfolding, and spreading by learning policies in a simulated environment and then transferring them into real world manipulators [8,16], or already performing all these tasks in simulation [17].…”

Section: A Cloth Manipulationmentioning

confidence: 99%

Semantic State Estimation in Cloth Manipulation Tasks

Tzelepis¹,

Aksoy²,

Borràs³

et al. 2022

Preprint

View full text Add to dashboard Cite

Understanding of deformable object manipulations such as textiles is a challenge due to the complexity and high dimensionality of the problem. Particularly, the lack of a generic representation of semantic states (e.g., crumpled, diagonally folded) during a continuous manipulation process introduces an obstacle to identify the manipulation type. In this paper, we aim to solve the problem of semantic state estimation in cloth manipulation tasks. For this purpose, we introduce a new large-scale fully-annotated RGB image dataset showing various human demonstrations of different complicated cloth manipulations. We provide a set of baseline deep networks and benchmark them on the problem of semantic state estimation using our proposed dataset. Furthermore, we investigate the scalability of our semantic state estimation framework in robot monitoring tasks of long and complex cloth manipulations. We also release the dataset and source code of the baseline models: https://github.com/BOBaraki/ state-estimation-for-continuous-cloth-manipulation

show abstract

Grasp-Oriented Fine-grained Cloth Segmentation without Real Supervision

Cited by 2 publications

References 17 publications

Salient Semantic Segmentation Based on RGB-D Camera for Robot Semantic Mapping

Salient Semantic Segmentation Based on RGB-D Camera for Robot Semantic Mapping

Semantic State Estimation in Cloth Manipulation Tasks

Contact Info

Product

Resources

About