GKNet: Grasp keypoint network for grasp candidates detection

Abstract

Contemporary grasp detection approaches employ deep learning to achieve robustness to sensor and object model uncertainty. The two dominant approaches design either grasp-quality scoring or anchor-based grasp recognition networks. This paper presents a different approach to grasp detection by treating it as keypoint detection in image-space. The deep network detects each grasp candidate as a pair of keypoints, convertible to the grasp representation g = { x, y, w, θ} T, rather than a triplet or quartet of corner points. Decreasing the detection difficulty by grouping keypoints into pairs boosts performance. To promote capturing dependencies between keypoints, a non-local module is incorporated into the network design. A final filtering strategy based on discrete and continuous orientation prediction removes false correspondences and further improves grasp detection performance. GKNet, the approach presented here, achieves a good balance between accuracy and speed on the Cornell and the abridged Jacquard datasets (96.9% and 98.39% at 41.67 and 23.26 fps). Follow-up experiments on a manipulator evaluate GKNet using four types of grasping experiments reflecting different nuisance sources: static grasping, dynamic grasping, grasping at varied camera angles, and bin picking. GKNet outperforms reference baselines in static and dynamic grasping experiments while showing robustness to varied camera viewpoints and moderate clutter. The results confirm the hypothesis that grasp keypoints are an effective output representation for deep grasp networks that provide robustness to expected nuisance factors.

Research Status of Robot Grab Detection Based on Vision

“…Combining pixel and channel attention, the features of the captured object are further highlighted by suppressing noise features. In 2022, Xu et al [12] proposed Grab Network (GKN) based on key point detection. Each candidate capture point in the network is detected as a pair of key points, and the difficulty of detection is reduced by grouping the key points into pairs.…”

Video based basketball shooting prediction and pose suggestion system

“…Ablation study: To better understand the benefits of All results for baselines adopted from original paper for reference, following [3].…”

“…Robotic grasping is a fundamental yet demanding problem, requiring both object perception as well as geometric reasoning based solely on sensor input. Past reasearchers simplified the problem by constraining the grasp poses into SE(2) space, assuming that the camera looks at the scene vertically from the top, and the gripper reaches perpendicularly to the support plane [1], [2], [3]. The restriction allows the planar grasp methods to represent grasps as simple oriented rectangles or keypoints in the image space, which permits directly adopting existing data-driven tools from computer vision tasks, such as object [4] or keypoint [5] detectors.…”

Learning Suction Graspability Considering Grasp Quality and Robot Reachability for Bin-Picking

“…In early studies, deep neural networks were used to directly predict the candidate grasp configurations without considering the grasp quality (Asif et al, 2018 ; Zhou X. et al, 2018 ; Xu et al, 2021 ). However, since there can be multiple grasp candidates for an object that has a complicated shape or multiple objects in a cluttered scene, learning graspablity is required for the planner to find the optimal grasp among the candidates.…”