Visual–tactile object recognition of a soft gripper based on faster Region-based Convolutional Neural Network and machining learning algorithm

Jiao, Chenlei; Lian, Binbin; Wang, Zhe; Song, Yimin; Sun, Tao

doi:10.1177/1729881420948727

Cited by 15 publications

(9 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…7). On average, tactile-based classification achieves an accuracy of 89.3%, which is comparable with performances of other sensorized soft hands [23], [24]. In comparison, pressurebased classification achieves only 73.0%, highlighting the superiority of our tactile sensor.…”

Section: Classification Resultsmentioning

confidence: 61%

A Low-Cost, Easy-to-Manufacture, Flexible, Multi-Taxel Tactile Sensor and its Application to In-Hand Object Recognition

Pannen¹,

Puhlmann²,

Brock³

2021

Preprint

View full text Add to dashboard Cite

Soft robotics is an emerging field that yields promising results for tasks that require safe and robust interactions with the environment or with humans, such as grasping, manipulation, and human-robot interaction. Soft robots rely on intrinsically compliant components and are difficult to equip with traditional, rigid sensors which would interfere with their compliance. We propose a highly flexible tactile sensor that is low-cost and easy to manufacture while measuring contact pressures independently from 14 taxels. The sensor is built from piezoresistive fabric for highly sensitive, continuous responses and from a custom-designed flexible printed circuit board which provides a high taxel density. From these taxels, location and intensity of contact with the sensor can be inferred. In this paper, we explain the design and manufacturing of the proposed sensor, characterize its input-output relation, evaluate its effects on compliance when equipped to the silicone-based pneumatic actuators of the soft robotic RBO Hand 2, and demonstrate that the sensor provides rich and useful feedback for learning-based in-hand object recognition.

show abstract

Section: Classification Resultsmentioning

confidence: 61%

A Low-Cost, Easy-to-Manufacture, Flexible, Multi-Taxel Tactile Sensor and its Application to In-Hand Object Recognition

Pannen¹,

Puhlmann²,

Brock³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Long short-term memory (LSTM) ( Xie and Zhong, 2016b ) is typically utilized to process the data and classify the objects from the SoftMax function ( Zuo et al, 2021 ). However, since the strain sensors are normally made for one axis detection, they are usually insufficient for detection and need to be used together with other sensors such as tactile ( Zuo et al, 2021 ) and vision ( Jiao et al, 2020 ) sensors. Tactile sensors are widely used for detecting objects ( Jiao et al, 2020 ; She et al, 2020 ; YangHan et al, 2020 ; Subad et al, 2021 ; Zuo et al, 2021 ; Deng et al, 2022 ).…”

Section: Methods and Recent Developmentsmentioning

confidence: 99%

“…However, since the strain sensors are normally made for one axis detection, they are usually insufficient for detection and need to be used together with other sensors such as tactile ( Zuo et al, 2021 ) and vision ( Jiao et al, 2020 ) sensors. Tactile sensors are widely used for detecting objects ( Jiao et al, 2020 ; She et al, 2020 ; YangHan et al, 2020 ; Subad et al, 2021 ; Zuo et al, 2021 ; Deng et al, 2022 ). They can be built and fabricated on a small scale and embedded into soft grippers.…”

Section: Methods and Recent Developmentsmentioning

confidence: 99%

“…To enable object detection/classification when grasping using soft grippers, the sensors are usually integrated on the soft fingers to perceive the grasping mode. To detect the deformation of each finger, strain sensors are implemented into the soft grippers (Elgeneidy et al, 2018;Jiao et al, 2020;Souri et al, 2020;Zuo et al, 2021). These sensors can detect the deformation of bending Frontiers in Robotics and AI frontiersin.org actuators so that the grasping pose of different fingers can be estimated.…”

Section: F Soft Gripping Technologymentioning

confidence: 99%

See 1 more Smart Citation

Learning-based robotic grasping: A review

2023

View full text Add to dashboard Cite

As personalization technology increasingly orchestrates individualized shopping or marketing experiences in industries such as logistics, fast-moving consumer goods, and food delivery, these sectors require flexible solutions that can automate object grasping for unknown or unseen objects without much modification or downtime. Most solutions in the market are based on traditional object recognition and are, therefore, not suitable for grasping unknown objects with varying shapes and textures. Adequate learning policies enable robotic grasping to accommodate high-mix and low-volume manufacturing scenarios. In this paper, we review the recent development of learning-based robotic grasping techniques from a corpus of over 150 papers. In addition to addressing the current achievements from researchers all over the world, we also point out the gaps and challenges faced in AI-enabled grasping, which hinder robotization in the aforementioned industries. In addition to 3D object segmentation and learning-based grasping benchmarks, we have also performed a comprehensive market survey regarding tactile sensors and robot skin. Furthermore, we reviewed the latest literature on how sensor feedback can be trained by a learning model to provide valid inputs for grasping stability. Finally, learning-based soft gripping is evaluated as soft grippers can accommodate objects of various sizes and shapes and can even handle fragile objects. In general, robotic grasping can achieve higher flexibility and adaptability, when equipped with learning algorithms.

show abstract

“…Jiao et al proposed a visual-tactile recognition method based on machine learning and multisensor information to recognize ordinary objects. e Kinect v2 was adopted for acquiring visual information, and bending and force sensors were embedded in the soft fingers to obtain tactile information [21]. Sun et al presented two accurate object classification methods based on tactile sensing information that used the extreme learning method (ELM) and deep dynamical systems (DDSs).…”

Section: Introductionmentioning

confidence: 99%

Fruit Classification Utilizing a Robotic Gripper with Integrated Sensors and Adaptive Grasping

Zhang

Lai

et al. 2021

Mathematical Problems in Engineering

View full text Add to dashboard Cite

As the core component of agricultural robots, robotic grippers are widely used for plucking, picking, and harvesting fruits and vegetables. Secure grasping is a severe challenge in agricultural applications because of the variation in the shape and hardness of agricultural products during maturation, as well as their variety and delicacy. In this study, a fruit identification method utilizing an adaptive gripper with tactile sensing and machine learning algorithms is reported. An adaptive robotic gripper is designed and manufactured to perform adaptive grasping. A tactile sensing information acquisition circuit is built, and force and bending sensors are integrated into the robotic gripper to measure the contact force distribution on the contact surface and the deformation of the soft fingers. A robotic manipulator platform is developed to collect the tactile sensing data in the grasping process. The performance of the random forest (RF), k-nearest neighbor (KNN), support vector classification (SVC), naive Bayes (NB), linear discriminant analysis (LDA), and ridge regression (RR) classifiers in identifying and classifying five types of fruits using the adaptive gripper is evaluated and compared. The RF classifier achieves the highest accuracy of 98%, while the accuracies of the other classifiers vary from 74% to 97%. The experiment illustrates that efficient and accurate fruit identification can be realized with the adaptive gripper and machine learning classifiers, and that the proposed method can provide a reference for controlling the grasping force and planning the robotic motion in the plucking, picking, and harvesting of fruits and vegetables.

show abstract

Visual–tactile object recognition of a soft gripper based on faster Region-based Convolutional Neural Network and machining learning algorithm

Cited by 15 publications

References 44 publications

A Low-Cost, Easy-to-Manufacture, Flexible, Multi-Taxel Tactile Sensor and its Application to In-Hand Object Recognition

A Low-Cost, Easy-to-Manufacture, Flexible, Multi-Taxel Tactile Sensor and its Application to In-Hand Object Recognition

Learning-based robotic grasping: A review

Fruit Classification Utilizing a Robotic Gripper with Integrated Sensors and Adaptive Grasping

Contact Info

Product

Resources

About