Knowledge-Based Digital Twin for Predicting Interactions in Human-Robot Collaboration

Tuli, Tadele Belay; Kohl, Linus; Chala, Sisay Adugna; Manns, Martin; Ansari, Fazel

doi:10.1109/etfa45728.2021.9613342

Cited by 15 publications

(12 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Various human motion tracking sensors developed in recent decades are able to provide accurate results. However, gaze tracking [30], facial temperature, and other unobtrusive and miniaturized psychological and physiological data sensors are continuously evolving, making sensing the mental status of humans still a point of contention [31].…”

Section: Human-focused Sensorsmentioning

confidence: 99%

Human-Centric Digital Twins in Industry: A Comprehensive Review of Enabling Technologies and Implementation Strategies

Asad

Khan

Khalid

et al. 2023

Sensors

View full text Add to dashboard Cite

The last decade saw the emergence of highly autonomous, flexible, re-configurable Cyber-Physical Systems. Research in this domain has been enhanced by the use of high-fidelity simulations, including Digital Twins, which are virtual representations connected to real assets. Digital Twins have been used for process supervision, prediction, or interaction with physical assets. Interaction with Digital Twins is enhanced by Virtual Reality and Augmented Reality, and Industry 5.0-focused research is evolving with the involvement of the human aspect in Digital Twins. This paper aims to review recent research on Human-Centric Digital Twins (HCDTs) and their enabling technologies. A systematic literature review is performed using the VOSviewer keyword mapping technique. Current technologies such as motion sensors, biological sensors, computational intelligence, simulation, and visualization tools are studied for the development of HCDTs in promising application areas. Domain-specific frameworks and guidelines are formed for different HCDT applications that highlight the workflow and desired outcomes, such as the training of AI models, the optimization of ergonomics, the security policy, task allocation, etc. A guideline and comparative analysis for the effective development of HCDTs are created based on the criteria of Machine Learning requirements, sensors, interfaces, and Human Digital Twin inputs.

show abstract

Section: Human-focused Sensorsmentioning

confidence: 99%

Human-Centric Digital Twins in Industry: A Comprehensive Review of Enabling Technologies and Implementation Strategies

Asad

Khan

Khalid

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…The structure and models were applied as sociable robots' autonomous proxemic behavior systems. Tuli et al ( 2021 ) provide a notion for semantic visualization of human activities and intent forecast in a domain knowledge-based semantic info hub utilizing a flexible task ontology interface. Bolotnikova ( 2021 ) study the subject of whole-body anthropomorphic robot postural planning in the setting of assistive physical HRI.…”

Section: Recent Advancements Of Application For Multi-modal Human–rob...mentioning

confidence: 99%

Recent advancements in multimodal human–robot interaction

Yang

et al. 2023

Front. Neurorobot.

View full text Add to dashboard Cite

Robotics have advanced significantly over the years, and human–robot interaction (HRI) is now playing an important role in delivering the best user experience, cutting down on laborious tasks, and raising public acceptance of robots. New HRI approaches are necessary to promote the evolution of robots, with a more natural and flexible interaction manner clearly the most crucial. As a newly emerging approach to HRI, multimodal HRI is a method for individuals to communicate with a robot using various modalities, including voice, image, text, eye movement, and touch, as well as bio-signals like EEG and ECG. It is a broad field closely related to cognitive science, ergonomics, multimedia technology, and virtual reality, with numerous applications springing up each year. However, little research has been done to summarize the current development and future trend of HRI. To this end, this paper systematically reviews the state of the art of multimodal HRI on its applications by summing up the latest research articles relevant to this field. Moreover, the research development in terms of the input signal and the output signal is also covered in this manuscript.

show abstract

“…On the other hand, multiple fields work on robot's understanding of the human capabilities, preferences, plans and goals. For example, recently Tuli et al [8] built an ontological-based system for human intention inference in assembly operations (as in current goal), Liu et al [1] claimed to improve task scheduling in shared workspace settings through introducing dynamic and stochastic representations of the human task performance model and Rudenko et al [9] surveys advances in pedestrian trajectory prediction. Architecturally speaking, classically the human was treated as an element of the environment, but recently the paradigm of considering human-robot collaboration using a multi-agent view is receiving more attention [10].…”

Section: Shared Task Representation In Human-robot Teamsmentioning

confidence: 99%

Shared Task Representation for Human–Robot Collaborative Navigation: The Collaborative Search Case

Dalmasso,

Domínguez-Vidal,

Torres-Rodríguez

et al. 2023

Int J of Soc Robotics

View full text Add to dashboard Cite

Recent research in Human Robot Collaboration (HRC) has spread and specialised in many sub-fields. Many show considerable advances, but the human–robot collaborative navigation (HRCN) field seems to be stuck focusing on implicit collaboration settings, on hypothetical or simulated task allocation problems, on shared autonomy or on having the human as a manager. This work takes a step forward by presenting an end-to-end system capable of handling real-world human–robot collaborative navigation tasks. This system makes use of the Social Reward Sources model (SRS), a knowledge representation to simultaneously tackle task allocation and path planning, proposes a multi-agent Monte Carlo Tree Search (MCTS) planner for human–robot teams, presents the collaborative search as a testbed for HRCN and studies the usage of smartphones for communication in this setting. The detailed experiments prove the viability of the approach, explore collaboration roles adopted by the human–robot team and test the acceptability and utility of different communication interface designs.

show abstract

Knowledge-Based Digital Twin for Predicting Interactions in Human-Robot Collaboration

Cited by 15 publications

References 44 publications

Human-Centric Digital Twins in Industry: A Comprehensive Review of Enabling Technologies and Implementation Strategies

Human-Centric Digital Twins in Industry: A Comprehensive Review of Enabling Technologies and Implementation Strategies

Recent advancements in multimodal human–robot interaction

Shared Task Representation for Human–Robot Collaborative Navigation: The Collaborative Search Case

Contact Info

Product

Resources

About