Virtual-joint based motion similarity criteria for human–robot kinematics mapping

Chen, Zhang; Wang, Ziwei; Liang, Rongjian; Liang, Bin; Zhang, Tao

doi:10.1016/j.robot.2019.103412

Cited by 23 publications

(7 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The problem of kinematic re-targeting is discussed in [3], while proposing a virtual-joint based similarity criteria for evaluation mapping performance. Two platforms were implemented to experimentally verify the flexibility and the effectiveness of the proposed metric, both using the MATLAB framework.…”

Section: Related Workmentioning

confidence: 99%

“…Two specific configurations achieved the same top accuracy value of 98.72% and thus outperforming the best CNNs. This is one of the most relevant results of our experiments, suggesting that, by exploiting a sequence of [5,5], [5,5]] 64 [120, 80] [0.5, 0.5] 736,170 96.5142 0.1280 [ [5,5], [5,5]] 32 [160, 80] [0.5, 0.5] 454,386 96.5142 0.1241 [ [5,5], [3,3]] 64 [160, 0] [0.5, 0.5] 1,063,074 96.4418 0.1327 [ [5,5], [3,3] [3,3], [5,5]] 128 50 [0, 0] 2,981,310 98.7225 0.0413 5], [3,3]] 64 250 [0.5, 0.5] 6,690,094 98.7119 0.0541 [ [3,3], [5,5]] 32 250 [0.5, 0.5] 3,477,454 98.7119 0.0503 [ [3,3], [5,5]] 32 50 [0, 0] 676,254 98.6908 0.0438 [ [5,5], [5,5]] 128 250 [0, 0] 11,032,558 98.6908 0.0428 [ [5,5], [3,3]] 32 100 [0.5, 0.5] 1,330,682 98.6908 0.0497 [ [3,3], [5,5]] 128 100 [0, 0] 5,571,610 98.6908 0.0433 [ [3,3], [5,5]...…”

Section: Hand State Classifiermentioning

confidence: 99%

See 1 more Smart Citation

User-oriented Natural Human-Robot Control with Thin-Plate Splines and LRCN

Lima,

Amaral,

Nascimento-Jr

et al. 2021

Preprint

View full text Add to dashboard Cite

We propose a real-time vision-based teleoperation approach for robotic arms that employs a single depth-based camera, exempting the user from the need for any wearable devices. By employing a natural user interface, this novel approach leverages the conventional fine-tuning control, turning it into a direct body pose capture process. The proposed approach is comprised of two main parts. The first is a nonlinear customizable pose mapping based on Thin-Plate Splines (TPS), to directly transfer human body motion to robotic arm motion in a nonlinear fashion, thus allowing matching dissimilar bodies with different workspace shapes and kinematic constraints. The second is a Deep Neural Network hand-state classifier based on Long-term Recurrent Convolutional Networks (LRCN) that exploits the temporal coherence of the acquired depth data. We validate, evaluate and compare our approach through both classical cross-validation experiments of the proposed hand state classifier; and user studies over a set of practical experiments involving variants of pick-and-place and manufacturing tasks. Results revealed that LRCN networks outperform single image Convolutional Neural Networks; and that users' learning curves were steep, thus allowing the successful completion of the proposed tasks. When compared to a previous approach, the TPS approach revealed no increase in task complexity and similar times of completion, while providing more precise operation in regions closer to workspace boundaries.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Hand State Classifiermentioning

confidence: 99%

User-oriented Natural Human-Robot Control with Thin-Plate Splines and LRCN

Lima,

Amaral,

Nascimento-Jr

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…A network-based communication channel isolates a human operator from a potentially hazard interaction environment, which is therefore regarded as the special case of cyber-physical systems. In this regard, teleoperation triggers the systemic revolution of human-in-the-loop operation [1], providing a universal platform to medical diagnosis [2] and fault diagnosis [3][4][5] for industrial applications, which has been widely applied to space robotics [6][7][8][9], medical surgery [10][11][12][13] and deep-sea exploration [14,15].…”

Section: Introductionmentioning

confidence: 99%

A Novel Training and Collaboration Integrated Framework for Human–Agent Teleoperation

Huang

Wang

Bai

et al. 2021

Sensors

Self Cite

View full text Add to dashboard Cite

Human operators have the trend of increasing physical and mental workloads when performing teleoperation tasks in uncertain and dynamic environments. In addition, their performances are influenced by subjective factors, potentially leading to operational errors or task failure. Although agent-based methods offer a promising solution to the above problems, the human experience and intelligence are necessary for teleoperation scenarios. In this paper, a truncated quantile critics reinforcement learning-based integrated framework is proposed for human–agent teleoperation that encompasses training, assessment and agent-based arbitration. The proposed framework allows for an expert training agent, a bilateral training and cooperation process to realize the co-optimization of agent and human. It can provide efficient and quantifiable training feedback. Experiments have been conducted to train subjects with the developed algorithm. The performances of human–human and human–agent cooperation modes are also compared. The results have shown that subjects can complete the tasks of reaching and picking and placing with the assistance of an agent in a shorter operational time, with a higher success rate and less workload than human–human cooperation.

show abstract

“…Teleoperation is also widely adopted for complex robotic manipulation tasks, as it provides better high-level policies and more intelligent decisions with enhanced adaptability and flexibility by virtue of using human intelligence. Traditional works mainly focused on the single-master-single-slave teleoperation framework [13]- [16]. Different from the aforementioned works, dual-arm configuration is more beneficial to facilitate complicated operation tasks.…”

Section: Introductionmentioning

confidence: 99%

Dual-arm Coordinated Manipulation for Object Twisting with Human Intelligence

Bai¹,

Zhang²,

Huang³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Robotic dual-arm twisting is a common but very challenging task in both industrial production and daily services, as it often requires dexterous collaboration, large scale of endeffector rotating, and good adaptivity for object manipulation. Meanwhile, safety and efficiency are preliminary concerns for robotic dual-arm coordinated manipulation. Thus, the normally adopted fully automated task execution approaches based on environmental perception and motion planning techniques are still inadequate and problematic for the arduous twisting tasks. To this end, this paper presents a novel strategy of the dualarm coordinated control for twisting manipulation based on the combination of optimized motion planning for one arm and real time telecontrol with human intelligence for the other. The analysis and simulation results showed it can achieve collision and singularity free for dual arms with enhanced dexterity, safety, and efficiency.

show abstract

Virtual-joint based motion similarity criteria for human–robot kinematics mapping

Cited by 23 publications

References 12 publications

User-oriented Natural Human-Robot Control with Thin-Plate Splines and LRCN

User-oriented Natural Human-Robot Control with Thin-Plate Splines and LRCN

A Novel Training and Collaboration Integrated Framework for Human–Agent Teleoperation

Dual-arm Coordinated Manipulation for Object Twisting with Human Intelligence

Contact Info

Product

Resources

About