In-air Knotting of Rope using Dual-Arm Robot based on Deep Learning

Suzuki, Kanata; Kanamura, Momomi; Suga, Yuki; Mori, Hiroki; Ogata, Tetsuya

doi:10.1109/iros51168.2021.9635954

Cited by 22 publications

(17 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, when a command with the same position and task completion time was input, the same motion with very little variation was generated without considering the size and shape of the object. Recently, methods using raw images have been studied [51], [52]. These studies add raw images to the NN input and learn to generate appropriate motions in response to changes in the position, and shape of the object.…”

Section: ) Resultsmentioning

confidence: 99%

“…However, the spatial information in this study is limited to the center position of the pancake at the beginning of the task, and the shape and size of the object were not considered. Therefore, our future work is to integrate the proposed method with a real-time image-based motion generation method [51] and a method that considers the shape and size of multiple objects [52] to expand the tasks that can be performed by the robot in space and time.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Imitation Learning for Nonprehensile Manipulation Through Self-Supervised Learning Considering Motion Speed

2022

View full text Add to dashboard Cite

Robots are expected to replace menial tasks such as housework. Some of these tasks include nonprehensile manipulation performed without grasping objects. Nonprehensile manipulation is very difficult because it requires considering the dynamics of environments and objects. Imitating complex behaviors requires a large number of human demonstrations. In this study, a self-supervised learning that considers motion speed to achieve variable speed for nonprehensile manipulation is proposed. The proposed method collects and fine-tunes only successful actions among the data obtained during autonomous operations. By fine-tuning the successful data using speed labels, the robot learns the dynamics among itself, its environment, and objects. We experimented with the task of scooping and transporting pancakes using the neural network model trained on 24 human-collected training data. The proposed method significantly improved the success rate from 40.2% to 85.7%, and succeeded more than 75% for other objects.

show abstract

Section: ) Resultsmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Imitation Learning for Nonprehensile Manipulation Through Self-Supervised Learning Considering Motion Speed

2022

View full text Add to dashboard Cite

show abstract

“…Two types of neural networks were used: a CNN-based deep autoencoder for image feature extraction and a full connected (FC) network for integrating image features and robot joint angles. Suzuki et al [4] enabled robots to manipulate ropes by two arms on the bases of the method of Yang et al [9]. In addition to the image and the joint angle of the robot, the proximity sensor was used to recognize the rope state.…”

Section: A Deep Learning-based Motion Learningmentioning

confidence: 99%

“…However, it is very challenging to make robots manipulate these flexible objects. Flexible object manipulations achieved by robots so far include folding clothes, tying ropes, and folding paper [1][2][3] [4]. For these tasks, it is important to predict how the object will be deformed by the robot's actions, and these tasks have been accomplished mainly using vision.…”

Section: Introductionmentioning

confidence: 99%

“…Therefore, methods based on deep learning with vision have been investigated for flexible object manipulation. Deep predictive learning, which learns to simultaneously predict motions and images to generate appropriate robot motions while predicting the behavior of flexible objects [4][9] [10], and reinforcement learning-based methods using real robots [11] [12] have been proposed. These methods are promising for contact-rich flexible object manipulation because they do not require a physical model of the target flexible object since they learn from the robot's motion data.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Contact-Rich Manipulation of a Flexible Object based on Deep Predictive Learning using Vision and Tactility

Ichiwara¹,

Itoh²,

Yamamoto³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We achieved contact-rich flexible object manipulation, which was difficult to control with vision alone. In the unzipping task we chose as a validation task, the gripper grasps the puller, which hides the bag state such as the direction and amount of deformation behind it, making it difficult to obtain information to perform the task by vision alone. Additionally, the flexible fabric bag state constantly changes during operation, so the robot needs to dynamically respond to the change. However, the appropriate robot behavior for all bag states is difficult to prepare in advance. To solve this problem, we developed a model that can perform contact-rich flexible object manipulation by real-time prediction of vision with tactility. We introduced a point-based attention mechanism for extracting image features, softmax transformation for predicting motions, and convolutional neural network for extracting tactile features. The results of experiments using a real robot arm revealed that our method can realize motions responding to the deformation of the bag while reducing the load on the zipper. Furthermore, using tactility improved the success rate from 56.7% to 93.3% compared with vision alone, demonstrating the effectiveness and high performance of our method.

show abstract

Behavioral learning of dish rinsing and scrubbing based on interruptive direct teaching considering assistance rate

Wakabayashi,

Kawaharazuka,

Okada

et al. 2024

Advanced Robotics

View full text Add to dashboard Cite

In-air Knotting of Rope using Dual-Arm Robot based on Deep Learning

Cited by 22 publications

References 32 publications

Imitation Learning for Nonprehensile Manipulation Through Self-Supervised Learning Considering Motion Speed

Imitation Learning for Nonprehensile Manipulation Through Self-Supervised Learning Considering Motion Speed

Contact-Rich Manipulation of a Flexible Object based on Deep Predictive Learning using Vision and Tactility

Behavioral learning of dish rinsing and scrubbing based on interruptive direct teaching considering assistance rate

Contact Info

Product

Resources

About