To grasp or not to grasp: an end-to-end deep-learning approach for predicting grasping failures in soft hands

Arapi, Visar; Zhang, Yujie; Averta, Giuseppe; Catalano, Manuel G.; Rus, Daniela; Santina, Cosimo Della; Bianchi, Matteo

doi:10.1109/robosoft48309.2020.9116041

Cited by 9 publications

(5 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A deep learning-based method has been developed to address the challenge of predicting whether the grasp will be successful in soft hands [186]. This framework utilizes two neural architectures: a classifier for the a-posteriori detection of failure events and a predictor that uses readings from Inertial Measurement Units (IMUs) to estimate object sliding.…”

Section: Machine Learning In Soft Handsmentioning

confidence: 99%

Anthropomorphic Soft Hand: Dexterity, Sensing, and Machine Learning

Wang,

Hao,

Liu

et al. 2024

Actuators

View full text Add to dashboard Cite

Humans possess dexterous hands that surpass those of other animals, enabling them to perform intricate, complex movements. Soft hands, known for their inherent flexibility, aim to replicate the functionality of human hands. This article provides an overview of the development processes and key directions in soft hand evolution. Starting from basic multi-finger grippers, these hands have made significant advancements in the field of robotics. By mimicking the shape, structure, and functionality of human hands, soft hands can partially replicate human-like movements, offering adaptability and operability during grasping tasks. In addition to mimicking human hand structure, advancements in flexible sensor technology enable soft hands to exhibit touch and perceptual capabilities similar to humans, enhancing their performance in complex tasks. Furthermore, integrating machine learning techniques has significantly promoted the advancement of soft hands, making it possible for them to intelligently adapt to a variety of environments and tasks. It is anticipated that these soft hands, designed to mimic human dexterity, will become a focal point in robotic hand development. They hold significant application potential for industrial flexible gripping solutions, medical rehabilitation, household services, and other domains, offering broad market prospects.

show abstract

Section: Machine Learning In Soft Handsmentioning

confidence: 99%

Anthropomorphic Soft Hand: Dexterity, Sensing, and Machine Learning

Wang,

Hao,

Liu

et al. 2024

Actuators

View full text Add to dashboard Cite

show abstract

“…Morrison et al [29] proposed a real-time generation method of robot grasping, and established the grasping convolution neural network (GG-CNN) to directly generate the grasping posture from the pixel-based depth image. Arapi et al [30] proposed a framework for soft robot arms grasping fault prediction based on deep learning. By training the network, the problem of fault prediction of flexible robot arms before grasping was solved and the occurrence of the fault was predicted.…”

Section: Air Suctionmentioning

confidence: 99%

Deep-Learning-Based Accurate Identification of Warehouse Goods for Robot Picking Operations

Liu

Zhao

et al. 2022

Sustainability

View full text Add to dashboard Cite

In order to explore the application of robots in intelligent supply-chain and digital logistics, and to achieve efficient operation, energy conservation, and emission reduction in the field of warehousing and sorting, we conducted research in the field of unmanned sorting and automated warehousing. Under the guidance of the theory of sustainable development, the ESG (Environmental Social Governance) goals in the social aspect are realized through digital technology in the storage field. In the picking process of warehousing, efficient and accurate cargo identification is the premise to ensure the accuracy and timeliness of intelligent robot operation. According to the driving and grasping methods of different robot arms, the image recognition model of arbitrarily shaped objects is established by using a convolution neural network (CNN) on the basis of simulating a human hand grasping objects. The model updates the loss function value and global step size by exponential decay and moving average, realizes the identification and classification of goods, and obtains the running dynamics of the program in real time by using visual tools. In addition, combined with the different characteristics of the data set, such as shape, size, surface material, brittleness, weight, among others, different intelligent grab solutions are selected for different types of goods to realize the automatic picking of goods of any shape in the picking list. Through the application of intelligent item grabbing in the storage field, it lays a foundation for the construction of an intelligent supply-chain system, and provides a new research perspective for cooperative robots (COBOT) in the field of logistics warehousing.

show abstract

“…Nevertheless, it is possible to apply CNNs for tactile sensor data processing. In [18], failures during grasping were predicted by the information from an array of Inertial Measurement Units. Gandarias at al.…”

Section: A Architecture Descriptionmentioning

confidence: 99%

DeepXPalm: Tilt and Position Rendering using Palm-worn Haptic Display and CNN-based Tactile Pattern Recognition

Cabrera

Oleg

Jonathan

et al. 2022

2022 IEEE Haptics Symposium (HAPTICS)

View full text Add to dashboard Cite

Telemanipulation of deformable objects requires high precision and dexterity from the users, which can be increased by kinesthetic and tactile feedback. However, the object shape can change dynamically, causing ambiguous perception of its alignment and hence errors in the robot positioning. Therefore, the tilt angle and position classification problem has to be solved to present a clear tactile pattern to the user. This work presents a telemanipulation system for plastic pipettes consisting of a multi-contact haptic device LinkGlide to deliver haptic feedback at the users' palm and two tactile sensors array embedded in the 2-finger Robotiq gripper. We propose a novel approach based on Convolutional Neural Networks (CNN) to detect the tilt and position while grasping deformable objects. The CNN generates a mask based on recognized tilt and position data to render further multi-contact tactile stimuli provided to the user during the telemanipulation. The study has shown that using the CNN algorithm and the preset mask, tilt, and position recognition by users is increased from 9.67% using the direct data to 82.5%.

show abstract

To grasp or not to grasp: an end-to-end deep-learning approach for predicting grasping failures in soft hands

Cited by 9 publications

References 27 publications

Anthropomorphic Soft Hand: Dexterity, Sensing, and Machine Learning

Anthropomorphic Soft Hand: Dexterity, Sensing, and Machine Learning

Deep-Learning-Based Accurate Identification of Warehouse Goods for Robot Picking Operations

DeepXPalm: Tilt and Position Rendering using Palm-worn Haptic Display and CNN-based Tactile Pattern Recognition

Contact Info

Product

Resources

About