Latency mitigation using applied HMMs for mixed reality-enhanced intuitive teleoperation in intelligent robotic welding

Abstract

Latency mitigation is crucial to increasing operational success, ease of use, and product quality in telemanipulation tasks when remotely guiding complex robotic systems. Hardware limitations have created a gap in performance optimization due to large teleoperation delays, which machine learning techniques could fill with lower time, improved performance, and reduced operating costs. Hidden Markov models (HMMs), in particular, have been explored to alleviate the issue due to their relative ease of use. A mixed reality-enhanced intuitive teleoperation framework for immersive and intuitive telerobotic welding is presented. The proposed system implements an HMM generative algorithm to learn and predict human-welder motion to enable a low-cost solution, combining smoothing and forecasting techniques to minimize robotic teleoperation time delay. The predicted welding motion system is simple to implement, can be used as a general solution to solve time delays, and is accurate. More specifically, it provides a 66% RMSE reduction compared to the application without HMM, which may be further optimized by up to 38%. Experiments show the HMM generative algorithm lets humans conduct tele-robot-assisted welding with better performance.

Effectiveness of IoT and VR for Real-Time Teleoperation of Industrial Robots

“…With the advantage of virtual reality, teleoperation robot platforms have become particularly useful for hazardous and obstructed environments, allowing digital transformation and improved scalability. Yunpeng Su et al [12] proposed virtual reality platform with haptic input device and Hidden Markov models algorithm to improve performance and intuitiveness in remote welding operation. Yuan Sun et al implement VR [62] to support construction-related tasks with mobile robots.…”

“…Finally, smooth and rapid interaction between the robot and the operator is critical. Insufficient or inconsistent connections can cause significant issues, such as delayed execution of user commands and, in extreme situations, command failure to act [10]. This problem has serious consequences because it can disrupt system functionality, reduce precision, and potentially cause hardware damage [11] [12].…”

Hidden Semi-Markov Models-Based Visual Perceptual State Recognition for Pilots

“…Wang et al [ 15 ] used the Gaussian mixture–hidden Markov model (G-HMM) to model the oculomotor scan pathway, providing a new approach to the screening, quantification, and assessment of disease. Yunpeng Su et al [ 16 ] used the HMM algorithm to make welding operations easier for robots in mixed-reality situations, reducing the time and cost even more than applications that did not use the HMM.…”

Research on Teleoperated Virtual Reality Human–Robot Five-Dimensional Collaboration System

“…With recent advancements in communication technologies like 5G and Wi-Fi 6, coupled with the development of the tactile internet, it is now possible to control and monitor remote robots in real-time using mixed reality [25], all within a low-latency, high-precision communication environment [26]. The integration of the tactile internet with communication technologies has addressed stability and transparency issues that were previously hindered by network problems.…”