In-Hand Object Recognition with Innervated Fiber Optic Spectroscopy for Soft Grippers

Hanson, Nathaniel; Hochsztein, Hillel; Vaidya, Akshay; Willick, Joel; Dorsey, Kristen L.; Padır, Taşkın

doi:10.1109/robosoft54090.2022.9762166

Cited by 7 publications

(3 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In prior experiments, we designed and implemented a gripper system to acquire recursive estimation models of grasped items as an items was grasped by a parallel-plate gripper Hanson et al (2022b). We have also applied spectroscopy to applications in soft robotics Hanson et al (2022a) and mobile robotics Hanson et al (2022c). In these studies we have found spectral signatures to be an extremely explainable metric for understanding abstract material types in the context of robotics.…”

Section: Spectroscopy In Robotsmentioning

confidence: 99%

Occluded object detection and exposure in cluttered environments with automated hyperspectral anomaly detection

2022

View full text Add to dashboard Cite

Cluttered environments with partial object occlusions pose significant challenges to robot manipulation. In settings composed of one dominant object type and various undesirable contaminants, occlusions make it difficult to both recognize and isolate undesirable objects. Spatial features alone are not always sufficiently distinct to reliably identify anomalies under multiple layers of clutter, with only a fractional part of the object exposed. We create a multi-modal data representation of cluttered object scenes pairing depth data with a registered hyperspectral data cube. Hyperspectral imaging provides pixel-wise Visible Near-Infrared (VNIR) reflectance spectral curves which are invariant in similar material types. Spectral reflectance data is grounded in the chemical-physical properties of an object, making spectral curves an excellent modality to differentiate inter-class material types. Our approach proposes a new automated method to perform hyperspectral anomaly detection in cluttered workspaces with the goal of improving robot manipulation. We first assume the dominance of a single material class, and coarsely identify the dominant, non-anomalous class. Next these labels are used to train an unsupervised autoencoder to identify anomalous pixels through reconstruction error. To tie our anomaly detection to robot actions, we then apply a set of heuristically-evaluated motion primitives to perturb and further expose local areas containing anomalies. The utility of this approach is demonstrated in numerous cluttered environments including organic and inorganic materials. In each of our four constructed scenarios, our proposed anomaly detection method is able to consistently increase the exposed surface area of anomalies. Our work advances robot perception for cluttered environments by incorporating multi-modal anomaly detection aided by hyperspectral sensing into detecting fractional object presence without need for laboriously curated labels.

show abstract

Section: Spectroscopy In Robotsmentioning

confidence: 99%

Occluded object detection and exposure in cluttered environments with automated hyperspectral anomaly detection

2022

View full text Add to dashboard Cite

show abstract

“…We utilize a StellarNet BLUE-Wave miniature VNIR spectrometer as its high SNR, 1000:1, provided consistent readings in prior robot grasping experiments [146], [147]. The system collects readings from wavelengths ranging from 350-1150 nm through a single input slit.…”

Section: Spectral Acquisition Systemmentioning

confidence: 99%

“…Our approach is grounded in observations that many organic and inorganic materials reflect distinct quantities of incident light across the spectrum allowing their identification by spectral signature. This work originally appeared in [147].…”

Section: Innervated Fiber Opticsmentioning

confidence: 99%

Material informed robotics - spectral perception for object identification and parameter inference

Hanson

View full text Add to dashboard Cite

show abstract

A Soft Robot with Three Dimensional Shape Sensing and Contact Recognition Multi-Modal Sensing via Tunable Soft Optical Sensors

McCandless,

Wise,

Russo

2023

2023 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

In-Hand Object Recognition with Innervated Fiber Optic Spectroscopy for Soft Grippers

Cited by 7 publications

References 25 publications

Occluded object detection and exposure in cluttered environments with automated hyperspectral anomaly detection

Occluded object detection and exposure in cluttered environments with automated hyperspectral anomaly detection

Material informed robotics - spectral perception for object identification and parameter inference

A Soft Robot with Three Dimensional Shape Sensing and Contact Recognition Multi-Modal Sensing via Tunable Soft Optical Sensors

Contact Info

Product

Resources

About