All You Need is LUV: Unsupervised Collection of Labeled Images using Invisible UV Fluorescent Indicators

Abstract

Large-scale semantic image annotation is a significant challenge for learning-based perception systems in robotics. Current approaches often rely on human labelers, which can be expensive, or simulation data, which can visually or physically differ from real data. This paper proposes Labels from UltraViolet (LUV), a novel framework that enables rapid, labeled data collection in real manipulation environments without human labeling. LUV uses transparent, ultravioletfluorescent paint with programmable ultraviolet LEDs to collect paired images of a scene in standard lighting and UV lighting to autonomously extract segmentation masks and keypoints via color segmentation. We apply LUV to a suite of diverse robot perception tasks to evaluate its labeling quality, flexibility, and data collection rate. Results suggest that LUV is 180-2500 times faster than a human labeler across the tasks. We show that LUV provides labels consistent with human annotations on unpainted test images. The networks trained on these labels are used to smooth and fold crumpled towels with 83% success rate and achieve 1.7mm position error with respect to human labels on a surgical needle pose estimation task. The low cost of LUV makes it ideal as a lightweight replacement for human labeling systems, with the one-time setup costs at $300 equivalent to the cost of collecting around 200 semantic segmentation labels on Amazon Mechanical Turk. Code, datasets, visualizations, and supplementary material can be found at https://sites. google.com/berkeley.edu/luv.

A Robust Deformable Linear Object Perception Pipeline in 3D: From Segmentation to Reconstruction

“…Thin Structure Segmentation: Thin structures and methods to segment them are common, from large scales such as road lane lines in autonomous driving [19], drone power line inspection [20], road networks in satellite image analysis [21], to small scales like tubular structures and blood vessels in medical image analysis [22], to cables, tubes, ropes, wires in household and industrial robotic manipulation tasks [16,11,23] which is the focus of this paper.…”

“…DLO Segmentation: Learning-based methods are widely adopted for DLO segmentation. Zanella et al [10] used an auto-augmented wire dataset with different backgrounds, Caporali et al [12] collected rendered DLO images with labels, and Thananjeyan et al [11] proposed a UAV labeling method on deformable objects to reduce human labeling effort.…”

“…Structured or simplified 2D working spaces, such as a table with uniform color [9] or 3D workspaces with simple backgrounds, are widely adopted by DLO tracking algorithms [5,4,6]. As for DLO segmentation in environments with different backgrounds, learningbased neural network segmentation architectures have been investigated [10,11,12]. These works show the importance of DLO data collection.…”

Occluded object detection and exposure in cluttered environments with automated hyperspectral anomaly detection

“…Outside of pure hyperspectral imaging, other works have proposed the use of infrared wavelengths of light to provide enhanced target detection Zhang et al (2021); Thananjeyan et al (2022). The successful use of extended wavelength information to identify humans and materials in scenes demonstrates the utility of the technique and its applicability to different problem spaces Mao et al (2022).…”