Deep learning of grasping detection for a robot used in sorting construction and demolition waste

Ku, Yuedong; Yang, Jianhong; Fang, Huaiying; Xiao, Wen; Zhuang, Jiangteng

doi:10.1007/s10163-020-01098-z

Cited by 49 publications

(18 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This section summarizes and classifies the robot hands and grippers in terms of the mechanism, material, and manipulation strategy, which are employed for sorting manipulation. Furthermore, robotic sorting manipulation methods [95], [96], [97], [98], [99], [100], [101] specific to the end-effectors are explored.…”

Section: A End-effectors and Manipulationmentioning

confidence: 99%

Challenges for Future Robotic Sorters of Mixed Industrial Waste: A Survey

Kiyokawa

Takamatsu

Koyanaka

2024

IEEE Trans. Automat. Sci. Eng.

View full text Add to dashboard Cite

To achieve recycling of mixed industrial waste toward an advanced sustainable society, waste sorting automation through robots is crucial and urgent. For this purpose, a robot is required to recognize the category, shape, pose, and condition of different waste items and manipulate them according to the category to be sorted. This survey considers three potential difficulties in the sorting automation: 1) End-effector: to robustly grasp and manipulate different waste items with dirt and deformations; 2) Sensor: to recognize the category, shape, and pose of existing objects to be manipulated and the wet and dirty conditions of their surfaces; and 3) Planner: to generate feasible and efficient sequences and trajectories. This survey includes 76 references to studies related to automatic waste sorting and 159 references to worldwide waste recycling attempts. This pioneering investigation reveals the possibility and limitations of conventional systems; thus, providing insights on open issues and potential technologies to achieve a robot-incorporated sorter for the chaotic mixed waste is one of its contributions. This paper further presents a system design policy for readers and discusses future advanced sorters, thereby contributing to the field of robotics and automation.Note to Practitioners-Most automated sorting systems operate for limited target waste items. This study is motivated by the automation of mixed industrial waste treatment facilities using advanced robotic sorters. Emerging advances and increasing functionalities of robot system components will widen system applicability and increase use cases in the chaotic mixed industrial waste domain. This paper surveys the research conducted to date, discusses open issues and potential approaches, and presents user guides that provide practitioners with a system design policy. The user guides created according to the strengths and weaknesses of each system configuration provide future researchers and developers with a useful a priori design policy that has been thus far validated on efficiency, quality, productivity, and reliability. A question-and-answer style guide Manuscript

show abstract

Section: A End-effectors and Manipulationmentioning

confidence: 99%

Challenges for Future Robotic Sorters of Mixed Industrial Waste: A Survey

Kiyokawa

Takamatsu

Koyanaka

2024

IEEE Trans. Automat. Sci. Eng.

View full text Add to dashboard Cite

show abstract

“…All these algorithms were adopted in solid waste classification shortly after having been launched. Relevant practices are presented in Table , and wastes were involved which included domestic wastes, CDW, and WEEE − and achieved inspired results. Faster R-CNN and Mask R-CNN have been prevailing because of high accuracy and detection speed.…”

Section: Development and Status Quo Of Sensor-based Waste Sorting Tec...mentioning

confidence: 99%

Sensor-Based Technologies in Effective Solid Waste Sorting: Successful Applications, Sensor Combination, and Future Directions

Zhao

2022

Environ. Sci. Technol.

View full text Add to dashboard Cite

The increase in global population and improvement of living standards have stirred up a continuous increase in solid waste generation, while simple incineration and landfilling bring about serious environmental and health concerns. In order to improve resource recovery and mitigate pollution, noncontacting and nondestructive sensor-based waste sorting systems are applied to enhance solid waste classification. In recent years, in addition to the rapid development of computer hardware, especially improvements of GPU computing capacity, complicated and efficient classification algorithms have emerged and been widely used in industrial sectors. These advances allow computers to process signals from sensors more quickly and accurately and to classify matters automatically. This article introduces widely applied sensor-based technologies in solid waste sorting and analyzes applicable conditions for each specific method. The latest developed algorithms are critically compared with competitive counterparts. Successful practices are described, and findings are highlighted. Though spectroscopic-based and vision-based waste classifications have achieved high performance in accuracy and detection speed, challenges and future directions can still provide wide development opportunities. Concretely, these opportunities generally comprise classification of indistinct plastics, application of the latest object detection algorithms, appropriate data set formulating, and sensor combination for multiple sorting tasks within a single system.

show abstract

“…Hyperspectral imaging has found widespread acceptance as a tool in machine vision for classification of fruits, waste products, and in many other organic and inorganic items Barnabé et al (2015); Ku et al (2021); Kwak et al (2021). These approaches demonstrate the diversity of applications the technology is relevant to, but lack generality to multiple robotic problems.…”

Section: Spectroscopy In Robotsmentioning

confidence: 99%

“…The first sensor, a Time of Flight (ToF) depth camera (Microsoft). At the nominal operating height of 0.7 m above the workcell surface, the ToF camera was estimated to have a systematic spatial error ≤ 1 mm, making it perfect for perceiving the presence of small objects in scene Kurillo et al (2022). The camera outputs registered RGB and depth images; the latter of which can easily be projected into 3D space…”

Section: Perceptionmentioning

confidence: 99%

“…Such tasks are monotonous and repetitive, thus making them perfect candidates for robots. Prior work has focused on specialized object classification models to address this problem Barnabé et al (2015); Erickson et al (2019); Ku et al (2021); Kwak et al (2021). However, application specific solutions can become resource intensive in data acquisition, labeling, and tuning.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Occluded object detection and exposure in cluttered environments with automated hyperspectral anomaly detection

2022

View full text Add to dashboard Cite

Cluttered environments with partial object occlusions pose significant challenges to robot manipulation. In settings composed of one dominant object type and various undesirable contaminants, occlusions make it difficult to both recognize and isolate undesirable objects. Spatial features alone are not always sufficiently distinct to reliably identify anomalies under multiple layers of clutter, with only a fractional part of the object exposed. We create a multi-modal data representation of cluttered object scenes pairing depth data with a registered hyperspectral data cube. Hyperspectral imaging provides pixel-wise Visible Near-Infrared (VNIR) reflectance spectral curves which are invariant in similar material types. Spectral reflectance data is grounded in the chemical-physical properties of an object, making spectral curves an excellent modality to differentiate inter-class material types. Our approach proposes a new automated method to perform hyperspectral anomaly detection in cluttered workspaces with the goal of improving robot manipulation. We first assume the dominance of a single material class, and coarsely identify the dominant, non-anomalous class. Next these labels are used to train an unsupervised autoencoder to identify anomalous pixels through reconstruction error. To tie our anomaly detection to robot actions, we then apply a set of heuristically-evaluated motion primitives to perturb and further expose local areas containing anomalies. The utility of this approach is demonstrated in numerous cluttered environments including organic and inorganic materials. In each of our four constructed scenarios, our proposed anomaly detection method is able to consistently increase the exposed surface area of anomalies. Our work advances robot perception for cluttered environments by incorporating multi-modal anomaly detection aided by hyperspectral sensing into detecting fractional object presence without need for laboriously curated labels.

show abstract

Deep learning of grasping detection for a robot used in sorting construction and demolition waste

Cited by 49 publications

References 26 publications

Challenges for Future Robotic Sorters of Mixed Industrial Waste: A Survey

Challenges for Future Robotic Sorters of Mixed Industrial Waste: A Survey

Sensor-Based Technologies in Effective Solid Waste Sorting: Successful Applications, Sensor Combination, and Future Directions

Occluded object detection and exposure in cluttered environments with automated hyperspectral anomaly detection

Contact Info

Product

Resources

About