Pregrasp Object Material Classification by a Novel Gripper Design with Integrated Spectroscopy

Abstract

Robots benefit from being able to classify objects they interact with or manipulate based on their material properties. This capability ensures fine manipulation of complex objects through proper grasp pose and force selection. Prior work has focused on haptic or visual processing to determine material type at grasp time. In this work, we introduce a novel parallel robot gripper design and a method for collecting spectral readings and visual images from within the gripper finger. We train a nonlinear Support Vector Machine (SVM) that can classify the material of the object about to be grasped through recursive estimation, with increasing confidence as the distance from the finger tips to the object decreases. In order to validate the hardware design and classification method, we collect samples from 16 real and fake fruit varieties (composed of polystyrene/plastic) resulting in a dataset containing spectral curves, scene images, and high-resolution texture images as the objects are grasped, lifted, and released. Our modeling method demonstrates an accuracy of 96.4% in classifying objects in a 32 class decision problem. This represents a performance improvement by 29.4% over the state of the art computer vision algorithms at distinguishing between visually similar materials. In contrast to prior work, our recursive estimation model accounts for increasing spectral signal strength and allows for decisions to be made as the gripper approaches an object. We conclude that spectroscopy is a promising sensing modality for enabling robots to not only classify grasped objects but also understand their underlying material composition.

Occluded object detection and exposure in cluttered environments with automated hyperspectral anomaly detection

“…This research effort builds on our prior experience with spectroscopy for robots. In prior experiments, we designed and implemented a gripper system to acquire recursive estimation models of grasped items as an items was grasped by a parallel-plate gripper Hanson et al (2022b). We have also applied spectroscopy to applications in soft robotics Hanson et al (2022a) and mobile robotics Hanson et al (2022c).…”

Material informed robotics - spectral perception for object identification and parameter inference

“…As we have observed in prior work, synthetic materials exhibit a large peak in spectral intensity around 900 nm [146]. The silicon-based photodetector ideally has quantum efficiency -the likelihood an incident photon triggering an electrical response -of at least 40% in the visible range.…”

“…We utilize a StellarNet BLUE-Wave miniature VNIR spectrometer as its high SNR, 1000:1, provided consistent readings in prior robot grasping experiments [146], [147]. The system collects readings from wavelengths ranging from 350-1150 nm through a single input slit.…”