GuLiM: A Hybrid Motion Mapping Technique for Teleoperation of Medical Assistive Robot in Combating the COVID-19 Pandemic

Lv, Honghao; Kong, Depeng; Pang, Gaoyang; Wang, Baicun; Zhao, Yu; Pang, Zhibo; Yang, Geng

doi:10.1109/tmrb.2022.3146621

Cited by 18 publications

(8 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The teleoperation system is a supported physical assistance solution for noncontact telehealth care. In the teleoperation system, the main technology is the motion mapping method, which was introduced previously [ 33 ]. With the two functional parts mentioned above, this assistive robot can be regarded as the second body of medical staff.…”

Section: Methodsmentioning

confidence: 99%

“…From the human side mentioned above, the position and posture data of the operator hands are obtained. Because the workspace of a human hand and the robotic manipulator is different, a previously proposed incremental pose-mapping strategy was used [ 33 ]. This method is mainly used to obtain the current human hand orientation and the increment of its position, and then to map it to the robot based on the current position of the robot.…”

Section: Methodsmentioning

confidence: 99%

“…The predefined different hand gestures stand for different robot motion control commands. Based on these, Lv et al [ 33 ] developed a hybrid mapping method of hand gestures and limb motion. Before the teleoperation begins, the operator does not need to assume the same posture as the robot arm.…”

Section: Methodsmentioning

confidence: 99%

See 2 more Smart Citations

A Medical Assistive Robot for Telehealth Care During the COVID-19 Pandemic: Development and Usability Study in an Isolation Ward

Wang¹,

Lv²,

Lu³

et al. 2023

JMIR Hum Factors

Self Cite

View full text Add to dashboard Cite

Background The COVID-19 pandemic is affecting the mental and emotional well-being of patients, family members, and health care workers. Patients in the isolation ward may have psychological problems due to long-term hospitalization, the development of the epidemic, and the inability to see their families. A medical assistive robot (MAR), acting as an intermediary of communication, can be deployed to address these mental pressures. Objective CareDo, a MAR with telepresence and teleoperation functions, was developed in this work for remote health care. The aim of this study was to investigate its practical performance in the isolation ward during the pandemic. Methods Two systems were integrated into the CareDo robot. For the telepresence system, a web real-time communications solution is used for the multiuser chat system and a convolutional neural network is used for expression recognition. For the teleoperation system, an incremental motion mapping method is used for operating the robot remotely. A clinical trial of this system was conducted at First Affiliated Hospital, Zhejiang University. Results During the clinical trials, tasks such as video chatting, emotion detection, and medical supplies delivery were performed via the CareDo robot. Seven voice commands were set for performing system wakeup, video chatting, and system exiting. Durations from 1 to 3 seconds of common commands were set to improve voice command detection. The facial expression was recorded 152 times for a patient in 1 day for the psychological intervention. The recognition accuracy reached 95% and 92.8% for happy and neutral expressions, respectively. Conclusions Patients and health care workers can use this MAR in the isolation ward for telehealth care during the COVID-19 pandemic. This can be a useful approach to break the chains of virus transmission and can also be an effective way to conduct remote psychological intervention.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

A Medical Assistive Robot for Telehealth Care During the COVID-19 Pandemic: Development and Usability Study in an Isolation Ward

Wang¹,

Lv²,

Lu³

et al. 2023

JMIR Hum Factors

Self Cite

View full text Add to dashboard Cite

show abstract

“…Using eight pre-trained neural networks, the system accurately captured and imitated 3D motions performed by a target human, enabling effective pose imitation and complex motion replication in the robot. Lv et al [101] presented a novel motion synchronization method called GuLiM for teleoperation of medical assistive robots, particularly in the context of combating the COVID-19 pandemic. Li et al [102] presented a multimodal mobile teleoperation system that integrated a vision-based hand pose regression network and an IMU-based arm tracking method.…”

Section: Perception Of Usersmentioning

confidence: 99%

Perception for Humanoid Robots

Roychoudhury,

Khorshidi,

Agrawal

et al. 2023

Curr Robot Rep

View full text Add to dashboard Cite

Purpose of Review The field of humanoid robotics, perception plays a fundamental role in enabling robots to interact seamlessly with humans and their surroundings, leading to improved safety, efficiency, and user experience. This scientific study investigates various perception modalities and techniques employed in humanoid robots, including visual, auditory, and tactile sensing by exploring recent state-of-the-art approaches for perceiving and understanding the internal state, the environment, objects, and human activities. Recent Findings Internal state estimation makes extensive use of Bayesian filtering methods and optimization techniques based on maximum a-posteriori formulation by utilizing proprioceptive sensing. In the area of external environment understanding, with an emphasis on robustness and adaptability to dynamic, unforeseen environmental changes, the new slew of research discussed in this study have focused largely on multi-sensor fusion and machine learning in contrast to the use of hand-crafted, rule-based systems. Human robot interaction methods have established the importance of contextual information representation and memory for understanding human intentions. Summary This review summarizes the recent developments and trends in the field of perception in humanoid robots. Three main areas of application are identified, namely, internal state estimation, external environment estimation, and human robot interaction. The applications of diverse sensor modalities in each of these areas are considered and recent significant works are discussed.

show abstract

“…The healthcare field requires technology for teleoperation of medical assistive robots with haptic and immersive video feedback. Although there have been studies on shared control methods and rehabilitation exercises [11,12], few have explored the synchronization of multiple haptic feedback while considering network delays. Our proposed system can address these gaps and has the potential to be applied in the healthcare field, for example, to perform rehabilitation exercises.…”

Section: Introductionmentioning

confidence: 99%

Locomotion-Based UAV Control Toward the Internet of Senses

Sehad

Cherif

Khadraoui

et al. 2023

IEEE Trans. Circuits Syst. II

View full text Add to dashboard Cite

The emergence of 5th-generation networks and the introduction of the ultra-low latency Internet, namely, tactile internet, by the International Telecommunication Union (ITU), has opened up a wide range of applications. Extended Reality (XR), holoportation, and remote control of machines are among the ones that would revolutionize the future of factories, smart cities, and digital healthcare. Virtual Reality (VR) technology provides users with highly realistic visual and auditory experiences, enabling a high sense of immersion and embodiment in virtual environments. In the real world, however, we use more senses than only vision and hearing to perceive our surroundings. Particularly, tactile sensation is the only bidirectional modality that enables us to perceive and interact with the objects and surfaces around us. This paper introduces a real-life testbed Unmanned Aerial Vehicle (UAV)-based system that leverages different technologies, including VR, 360 • video streaming over 4G/5G, and edge computing to enable 6 Degrees of Freedom (6DoF) view with haptic feedback for a maximized immersion. The demonstration videos of the testbed are made publicly available on the following links: Link1 and Link2.

show abstract

GuLiM: A Hybrid Motion Mapping Technique for Teleoperation of Medical Assistive Robot in Combating the COVID-19 Pandemic

Cited by 18 publications

References 46 publications

A Medical Assistive Robot for Telehealth Care During the COVID-19 Pandemic: Development and Usability Study in an Isolation Ward

A Medical Assistive Robot for Telehealth Care During the COVID-19 Pandemic: Development and Usability Study in an Isolation Ward

Perception for Humanoid Robots

Locomotion-Based UAV Control Toward the Internet of Senses

Contact Info

Product

Resources

About