Garbage Collection and Sorting with a Mobile Manipulator using Deep Learning and Whole-Body Control

Liu, Jingyi; Balatti, Pietro; Ellis, Kirsty; Hadjivelichkov, Denis; Stoyanov, Danail; Ajoudani, Arash; Kanoulas, Dimitrios

doi:10.1109/humanoids47582.2021.9555800

Cited by 19 publications

(10 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Environmental factors such as variation in weather and lighting conditions as well as background and scenery will also be addressed in future work Frontiers in Robotics and AI frontiersin.org through more advanced and complex computer vision algorithms. Other networks such as YOLACT which was used in (Liu et al, 2021) will also be tested to increase the control frequency of the visual-servoing approach such that faster-moving objects can be tracked and objects can be grasped faster. Further future work includes mounting the robot on a mobile robotic platform such that it can be autonomously deployed in the field.…”

Section: Discussionmentioning

confidence: 99%

“…The LitterBot also only requires 2D images for picking and binning , unlike prior works which require 3D point cloud images ( Lukka et al, 2014 ; Raptopoulos et al, 2020 ; Liu et al, 2021 ) for planning appropriate grasp movements. Our method circumvents this through the use of a soft gripper which greatly simplifies the control complexity without sacrificing performance.…”

Section: Discussionmentioning

confidence: 99%

“…Bai et al (2018) presented a novel garbage pickup robot tested on grass using a learning-based object segmentation algorithm. Liu et al (2021) developed a comprehensive system that uses deep learning for object segmentation and classification Frontiers in Robotics and AI frontiersin.org of different classes. Incorporated with a mobile robot, a grasp localization method to identify a suitable grasp pose to pick the garbage from the ground was also developed.…”

Section: Open Access Edited Bymentioning

confidence: 99%

See 2 more Smart Citations

Autonomous detection and sorting of litter using deep learning and soft robotic grippers

et al. 2022

View full text Add to dashboard Cite

Road infrastructure is one of the most vital assets of any country. Keeping the road infrastructure clean and unpolluted is important for ensuring road safety and reducing environmental risk. However, roadside litter picking is an extremely laborious, expensive, monotonous and hazardous task. Automating the process would save taxpayers money and reduce the risk for road users and the maintenance crew. This work presents LitterBot, an autonomous robotic system capable of detecting, localizing and classifying common roadside litter. We use a learning-based object detection and segmentation algorithm trained on the TACO dataset for identifying and classifying garbage. We develop a robust modular manipulation framework by using soft robotic grippers and a real-time visual-servoing strategy. This enables the manipulator to pick up objects of variable sizes and shapes even in dynamic environments. The robot achieves greater than 80% classified picking and binning success rates for all experiments; which was validated on a wide variety of test litter objects in static single and cluttered configurations and with dynamically moving test objects. Our results showcase how a deep model trained on an online dataset can be deployed in real-world applications with high accuracy by the appropriate design of a control framework around it.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Open Access Edited Bymentioning

confidence: 99%

See 1 more Smart Citation

Autonomous detection and sorting of litter using deep learning and soft robotic grippers

et al. 2022

View full text Add to dashboard Cite

show abstract

“…The domain names of the recycling groups are arranged in descending order from the left in the same manner. WEEE [59], [60], garbage in public spaces [61], [62], [63], [64], [65], [66], [67], [68], and household waste [69], [70], [71], [72], [73] were included.…”

Section: B Search and Collection Strategymentioning

confidence: 99%

Challenges for Future Robotic Sorters of Mixed Industrial Waste: A Survey

Kiyokawa

Takamatsu

Koyanaka

2024

IEEE Trans. Automat. Sci. Eng.

View full text Add to dashboard Cite

To achieve recycling of mixed industrial waste toward an advanced sustainable society, waste sorting automation through robots is crucial and urgent. For this purpose, a robot is required to recognize the category, shape, pose, and condition of different waste items and manipulate them according to the category to be sorted. This survey considers three potential difficulties in the sorting automation: 1) End-effector: to robustly grasp and manipulate different waste items with dirt and deformations; 2) Sensor: to recognize the category, shape, and pose of existing objects to be manipulated and the wet and dirty conditions of their surfaces; and 3) Planner: to generate feasible and efficient sequences and trajectories. This survey includes 76 references to studies related to automatic waste sorting and 159 references to worldwide waste recycling attempts. This pioneering investigation reveals the possibility and limitations of conventional systems; thus, providing insights on open issues and potential technologies to achieve a robot-incorporated sorter for the chaotic mixed waste is one of its contributions. This paper further presents a system design policy for readers and discusses future advanced sorters, thereby contributing to the field of robotics and automation.Note to Practitioners-Most automated sorting systems operate for limited target waste items. This study is motivated by the automation of mixed industrial waste treatment facilities using advanced robotic sorters. Emerging advances and increasing functionalities of robot system components will widen system applicability and increase use cases in the chaotic mixed industrial waste domain. This paper surveys the research conducted to date, discusses open issues and potential approaches, and presents user guides that provide practitioners with a system design policy. The user guides created according to the strengths and weaknesses of each system configuration provide future researchers and developers with a useful a priori design policy that has been thus far validated on efficiency, quality, productivity, and reliability. A question-and-answer style guide Manuscript

show abstract

“…Sequential navigation and manipulation: Due to the difficulties of planning in the conjoint space of the mobile manipulator base and arm, many existing approaches restrict themselves to sequential movements of the base followed by static manipulations with the arm. This decomposition has been popular across approaches based on reachability [8], planning [1], [19], [20], impedance control [21], and reinforcement learning [14], [22].…”

Section: Related Workmentioning

confidence: 99%

N$^2$M$^2$: Learning Navigation for Arbitrary Mobile Manipulation Motions in Unseen and Dynamic Environments

Honerkamp¹,

Welschehold²,

Valada³

2022

Preprint

View full text Add to dashboard Cite

Despite its importance in both industrial and service robotics, mobile manipulation remains a significant challenge as it requires a seamless integration of end-effector trajectory generation with navigation skills as well as reasoning over long-horizons. Existing methods struggle to control the large configuration space, and to navigate dynamic and unknown environments. In previous work, we proposed to decompose mobile manipulation tasks into a simplified motion generator for the end-effector in task space and a trained reinforcement learning agent for the mobile base to account for kinematic feasibility of the motion. In this work, we introduce Neural Navigation for Mobile Manipulation (N 2 M 2 ) which extends this decomposition to complex obstacle environments and enables it to tackle a broad range of tasks in real world settings. The resulting approach can perform unseen, long-horizon tasks in unexplored environments while instantly reacting to dynamic obstacles and environmental changes. At the same time, it provides a simple way to define new mobile manipulation tasks. We demonstrate the capabilities of our proposed approach in extensive simulation and real-world experiments on multiple kinematically diverse mobile manipulators. Code and videos are publicly available at http://mobile-rl.cs.uni-freiburg.de.

show abstract

Garbage Collection and Sorting with a Mobile Manipulator using Deep Learning and Whole-Body Control

Cited by 19 publications

References 21 publications

Autonomous detection and sorting of litter using deep learning and soft robotic grippers

Autonomous detection and sorting of litter using deep learning and soft robotic grippers

Challenges for Future Robotic Sorters of Mixed Industrial Waste: A Survey

N$^2$M$^2$: Learning Navigation for Arbitrary Mobile Manipulation Motions in Unseen and Dynamic Environments

Contact Info

Product

Resources

About