A fast detection and grasping method for mobile manipulator based on improved faster R-CNN

Zhang, Hui; Tan, Jinwen; Zhao, Chenyang; Liang, Zhicong; Liu, Li; Zhong, Huicai

doi:10.1108/ir-07-2019-0150

Cited by 10 publications

(5 citation statements)

References 30 publications

(50 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The ultimate goal of mobile manipulation is to operate the target object. Although methods for manipulating stationary objects or movement-known objects have been well studied [ 40 , 48 ], there are still great challenges for manipulating randomly placed and moving objects. Therefore, we use the object-detection model to detect and locate the object, and use the visual servo technology to control the manipulator to perform a tracking grab on the moving object.…”

Section: Methodsmentioning

confidence: 99%

Mobile Manipulation Integrating Enhanced AMCL High-Precision Location and Dynamic Tracking Grasp

Zhou

Chou

Tuo

et al. 2020

Sensors

View full text Add to dashboard Cite

Mobile manipulation, which has more flexibility than fixed-base manipulation, has always been an important topic in the field of robotics. However, for sophisticated operation in complex environments, efficient localization and dynamic tracking grasp still face enormous challenges. To address these challenges, this paper proposes a mobile manipulation method integrating laser-reflector-enhanced adaptive Monte Carlo localization (AMCL) algorithm and a dynamic tracking and grasping algorithm. First, by fusing the information of laser-reflector landmarks to adjust the weight of particles in AMCL, the localization accuracy of mobile platforms can be improved. Second, deep-learning-based multiple-object detection and visual servo are exploited to efficiently track and grasp dynamic objects. Then, a mobile manipulation system integrating the above two algorithms into a robotic with a 6-degrees-of-freedom (DOF) operation arm is implemented in an indoor environment. Technical components, including localization, multiple-object detection, dynamic tracking grasp, and the integrated system, are all verified in real-world scenarios. Experimental results demonstrate the efficacy and superiority of our method.

show abstract

Section: Methodsmentioning

confidence: 99%

Mobile Manipulation Integrating Enhanced AMCL High-Precision Location and Dynamic Tracking Grasp

Zhou

Chou

Tuo

et al. 2020

Sensors

View full text Add to dashboard Cite

show abstract

“…In contrast, a recent approach that gives state-of-the-art results relies on deep learning and convolution neural networks (CNNs) (Javidi and Jampour, 2020). Faster R- CNN Ren et al (2017), as a significant approach with successful applications (Zhang et al, 2020), introduces a region proposal network (RPN), which is a fully CNN that predicts object bounds and objectness scores.…”

Section: Object Detectionmentioning

confidence: 99%

An autonomous vision-based shelf-reader robot using faster R-CNN

Jampour

KarimiSardar²,

Estakhroyeh

2021

View full text Add to dashboard Cite

Purpose The purpose of this study is to design, program and implement an intelligent robot for shelf-reading. an essential task in library maintenance is shelf-reading, which refers to the process of checking the disciplines of books based on their call numbers to ensure that they are correctly shelved. Shelf-reading is a routine yet challenging task for librarians, as it involves controlling call numbers on the scale of thousands of books promptly. Design/methodology/approach Leveraging the strength of autonomous robots in handling repetitive tasks, this paper introduces a novel vision-based shelf-reader robot, called \emph{Pars} and demonstrate its effectiveness in accomplishing shelf-reading tasks. Also, this paper proposes a novel supervised approach to power the vision system of \emph{Pars}, allowing it to handle motion blur on images captured while it moves. An approach based on Faster R-CNN is also incorporated into the vision system, allowing the robot to efficiently detect the region of interest for retrieving a book’s information. Findings This paper evaluated the robot’s performance in a library with $120,000 books and discovered problems such as missing and misplaced books. Besides, this paper introduces a new challenging data set of blurred barcodes free publicly available for similar research studies. Originality/value The robot is equipped with six parallel cameras, which enable it to check books and decide moving paths. Through its vision-based system, it is also capable of routing and tracking paths between bookcases in a library and it can also turn around bends. Moreover, \emph{Pars} addresses the blurred barcodes, which may appear because of its motion.

show abstract

“…Studies using the RGB-D camera approach are presented in [6], [7], [8], [9], and [10]. Generally, the approach is similar to estimating pose using an RGB camera.…”

Section: Introductionmentioning

confidence: 99%

“…However, point cloud processing is beside this stream to extract the 2D object regions. One technique is presented in [10], which is not needed depth image post-processing. The depth image is used directly to measure the object's distance from the camera.…”

Section: Introductionmentioning

confidence: 99%

Object Detection and Pose Estimation using Rotatable Object Detector DRBox-v2 for Bin-Picking Robot

Jamzuri,

Pinandita,

Analia

et al. 2023

Proceedings of the 5th International Conference on Applied Engineering, ICAE 2022, 5 October 2022, Batam, Indonesia

View full text Add to dashboard Cite

This research aims to identify and estimate the object's pose to support bin-picking robot perception. In this research, we proposed the usage of the ArUco marker as a visual landmark of the detection area. Furthermore, the image of the detection area is processed by rotatable object detector DRBox-v2 to get the object's position and orientation in the camera frame. In the final process, the resulting DRBox-v2 position and orientation are transformed into a twodimensional world coordinate as the final estimated pose. Based on the experimental result, the object detection yields an Average Precision (AP) of 0.54 while a threshold score of 0.5 is used. As the pose estimation result, the proposed method yields an average position error of 0.21 cm and a maximum position error of 0.28 cm. For the orientation error, the system achieves a maximum orientation error of about 1.23 degrees with an average orientation error of 0.58 degrees. This research contributes to the possibility of camera usage and end-to-end deep learning detector supporting bin-picking research.

show abstract

A fast detection and grasping method for mobile manipulator based on improved faster R-CNN

Cited by 10 publications

References 30 publications

Mobile Manipulation Integrating Enhanced AMCL High-Precision Location and Dynamic Tracking Grasp

Mobile Manipulation Integrating Enhanced AMCL High-Precision Location and Dynamic Tracking Grasp

An autonomous vision-based shelf-reader robot using faster R-CNN

Object Detection and Pose Estimation using Rotatable Object Detector DRBox-v2 for Bin-Picking Robot

Contact Info

Product

Resources

About