Efficient and accurate segmentation of unseen objects is crucial for robotic manipulation. However, it remains challenging due to over-or under-segmentation. Although existing refinement methods can enhance the segmentation quality, they fix only minor boundary errors or are not sufficiently fast. In this work, we propose INSTAnce Boundary Explicit Error Estimation and Refinement (INSTA-BEEER), a novel refinement model that allows for adding and deleting instances and sharpening boundaries. Leveraging an error-estimationthen-refinement scheme, the model first estimates the pixelwise boundary explicit errors: true positive, true negative, false positive, and false negative pixels of the instance boundary in the initial segmentation. It then refines the initial segmentation using these error estimates as guidance. Experiments show that the proposed model significantly enhances segmentation, achieving state-of-the-art performance. Furthermore, with a fast runtime (less than 0.1 s), the model consistently improves performance across various initial segmentation methods, making it highly suitable for practical robotic applications.
Instance-aware segmentation of unseen objects is essential for a robotic system in an unstructured environment. Although previous works achieved encouraging results, they were limited to segmenting the only visible regions of unseen objects. For robotic manipulation in a cluttered scene, amodal perception is required to handle the occluded objects behind others. This paper addresses Unseen Object Amodal Instance Segmentation (UOAIS) to detect 1) visible masks, 2) amodal masks, and 3) occlusions on unseen object instances. For this, we propose a Hierarchical Occlusion Modeling (HOM) scheme designed to reason about the occlusion by assigning a hierarchy to a feature fusion and prediction order. We evaluated our method on three benchmarks (tabletop, indoors, and bin environments) and achieved state-of-the-art (SOTA) performance. Robot demos for picking up occluded objects, codes, and datasets are available at https://sites.google.com/view/uoais.
of AI technology to generate and validate the task plan for assembling furniture in the real and virtual environment by understanding the unstructured multi-modal information from the assembly manual) ABSTRACT Herpes zoster (HZ) is a common cutaneous disease affecting one out of five people; hence, early diagnosis of HZ is crucial as it can progress to chronic pain syndrome if antiviral treatment is not provided within 72 hr. Mobile diagnosis of HZ with the assistance of artificial intelligence can prevent neuropathic pain while reducing clinicians' fatigue and diagnosis cost. However, the clinical images captured from daily mobile devices likely contain visual corruptions, such as motion blur and noise, which can easily mislead the automated system. Hence, this paper aims to train a robust and mobile deep neural network (DNN) that can distinguish HZ from other skin diseases using user-submitted images. To enhance robustness while retaining low computational cost, we propose a knowledge distillation from ensemble via curriculum training (KDE-CT) wherein a student network learns from a stronger teacher network progressively. We established skin diseases dataset for HZ diagnosis and evaluated the robustness against 75 types of corruption. A total of 13 different DNNs was evaluated on both clean and corrupted images. The experiment result shows that the proposed KDE-CT significantly improves corruption robustness when compared with other methods. Our trained MobileNetV3-Small achieved more robust performance (93.5% overall accuracy, 67.6 mean corruption error) than the DNN ensemble with smaller computation (549x fewer multiply-and-accumulate operations), which makes it suitable for mobile skin lesion analysis.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.