Depth Image–Based Deep Learning of Grasp Planning for Textureless Planar-Faced Objects in Vision-Guided Robotic Bin-Picking

Jiang, Ping; Ishihara, Yoshiyuki; Sugiyama, Nobukatsu; Oaki, Junji; Tokura, Seiji; Sugahara, Atsushi; Ogawa, Akihito

doi:10.3390/s20030706

Cited by 35 publications

(20 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With the development of deep learning, some researchers try in the direction of learning-based object-agnostic sampling methods using neural networks. Jiang et al (2020) proposes a deep convolutional neural network (DCNN) to predict the set of grasp points from the input depth-image. Inspired by Varley et al (2015) that obtains grasps on pixels, Morrison et al (2018a) presents a Generative Grasping CNN (GG-CNN) which generates candidates immediately on pixelwise.…”

Section: Object-agnostic Samplingmentioning

confidence: 99%

Robotics Dexterous Grasping: The Methods Based on Point Cloud and Deep Learning

et al. 2021

View full text Add to dashboard Cite

Dexterous manipulation, especially dexterous grasping, is a primitive and crucial ability of robots that allows the implementation of performing human-like behaviors. Deploying the ability on robots enables them to assist and substitute human to accomplish more complex tasks in daily life and industrial production. A comprehensive review of the methods based on point cloud and deep learning for robotics dexterous grasping from three perspectives is given in this paper. As a new category schemes of the mainstream methods, the proposed generation-evaluation framework is the core concept of the classification. The other two classifications based on learning modes and applications are also briefly described afterwards. This review aims to afford a guideline for robotics dexterous grasping researchers and developers.

show abstract

Section: Object-agnostic Samplingmentioning

confidence: 99%

Robotics Dexterous Grasping: The Methods Based on Point Cloud and Deep Learning

et al. 2021

View full text Add to dashboard Cite

show abstract

“…The final goal is always determination of robot gripper coordinates, corresponding to appropriate grasp points. There are even examples of research aimed at grasp point determination without prior object segmentation [ 44 ]. Therefore, the object identification rules in extended VCD format contain not only conditions allowing to identify the objects to be manipulated by robot, but also the rules for calculation of “object parameters” (in the above example those rules are contained in lines 8 through 10).…”

Section: Integration Of Voice Command Description Format (Vcd) Andmentioning

confidence: 99%

Integration of Industrially-Oriented Human-Robot Speech Communication and Vision-Based Object Recognition

Rogowski

Bieliszczuk

Rapcewicz

2020

Sensors

View full text Add to dashboard Cite

This paper presents a novel method for integration of industrially-oriented human-robot speech communication and vision-based object recognition. Such integration is necessary to provide context for task-oriented voice commands. Context-based speech communication is easier, the commands are shorter, hence their recognition rate is higher. In recent years, significant research was devoted to integration of speech and gesture recognition. However, little attention was paid to vision-based identification of objects in industrial environment (like workpieces or tools) represented by general terms used in voice commands. There are no reports on any methods facilitating the abovementioned integration. Image and speech recognition systems usually operate on different data structures, describing reality on different levels of abstraction, hence development of context-based voice control systems is a laborious and time-consuming task. The aim of our research was to solve this problem. The core of our method is extension of Voice Command Description (VCD) format describing syntax and semantics of task-oriented commands, as well as its integration with Flexible Editable Contour Templates (FECT) used for classification of contours derived from image recognition systems. To the best of our knowledge, it is the first solution that facilitates development of customized vision-based voice control applications for industrial robots.

show abstract

“…The one-stage methods are usually faster and suitable for real-time applications, but the overall recognition accuracy may be lower than the two-stage methods. Object pose estimation is commonly based on RGB-D image [ 11 ] or 3D model [ 12 , 13 ] to calculate the 3D pose of object. Manuelli et al [ 14 ] proposed the keypoint [ 15 ] method, which is used to detect human skeleton and find the relative keypoints, and combined it with local dense geometric information from a point cloud.…”

Section: Introductionmentioning

confidence: 99%

Manipulation Planning for Object Re-Orientation Based on Semantic Segmentation Keypoint Detection

Wong

Yeh

Liu

et al. 2021

Sensors

View full text Add to dashboard Cite

In this paper, a manipulation planning method for object re-orientation based on semantic segmentation keypoint detection is proposed for robot manipulator which is able to detect and re-orientate the randomly placed objects to a specified position and pose. There are two main parts: (1) 3D keypoint detection system; and (2) manipulation planning system for object re-orientation. In the 3D keypoint detection system, an RGB-D camera is used to obtain the information of the environment and can generate 3D keypoints of the target object as inputs to represent its corresponding position and pose. This process simplifies the 3D model representation so that the manipulation planning for object re-orientation can be executed in a category-level manner by adding various training data of the object in the training phase. In addition, 3D suction points in both the object’s current and expected poses are also generated as the inputs of the next operation stage. During the next stage, Mask Region-Convolutional Neural Network (Mask R-CNN) algorithm is used for preliminary object detection and object image. The highest confidence index image is selected as the input of the semantic segmentation system in order to classify each pixel in the picture for the corresponding pack unit of the object. In addition, after using a convolutional neural network for semantic segmentation, the Conditional Random Fields (CRFs) method is used to perform several iterations to obtain a more accurate result of object recognition. When the target object is segmented into the pack units of image process, the center position of each pack unit can be obtained. Then, a normal vector of each pack unit’s center points is generated by the depth image information and pose of the object, which can be obtained by connecting the center points of each pack unit. In the manipulation planning system for object re-orientation, the pose of the object and the normal vector of each pack unit are first converted into the working coordinate system of the robot manipulator. Then, according to the current and expected pose of the object, the spherical linear interpolation (Slerp) algorithm is used to generate a series of movements in the workspace for object re-orientation on the robot manipulator. In addition, the pose of the object is adjusted on the z-axis of the object’s geodetic coordinate system based on the image features on the surface of the object, so that the pose of the placed object can approach the desired pose. Finally, a robot manipulator and a vacuum suction cup made by the laboratory are used to verify that the proposed system can indeed complete the planned task of object re-orientation.

show abstract

Depth Image–Based Deep Learning of Grasp Planning for Textureless Planar-Faced Objects in Vision-Guided Robotic Bin-Picking

Cited by 35 publications

References 47 publications

Robotics Dexterous Grasping: The Methods Based on Point Cloud and Deep Learning

Robotics Dexterous Grasping: The Methods Based on Point Cloud and Deep Learning

Integration of Industrially-Oriented Human-Robot Speech Communication and Vision-Based Object Recognition

Manipulation Planning for Object Re-Orientation Based on Semantic Segmentation Keypoint Detection

Contact Info

Product

Resources

About