Bringing a Natural Language-enabled Virtual Assistant to Industrial Mobile Robots for Learning, Training and Assistance of Manufacturing Tasks

Li, Chen; Hansen, Andreas Kornmaaler; Chrysostomou, Dimitrios; Bøgh, Simon; Madsen, Ole

doi:10.1109/sii52469.2022.9708757

“…In this work, we demonstrated a natural language-enabled VA, Max, that integrates with the LH8 robot platform to conduct multiple Learning, Training, and Assistance scenarios in a real industrial environment. In an expansion of our prior work [10], we conducted user studies with an emphasis on user feedback on satisfaction and usability of our system.…”

Section: Discussionmentioning

confidence: 99%

“…The last FFNN takes the results of the BERT model as input and outputs a possible user's intent (X intent ) and slots (X slot ). The model is trained and validated on a dialogue dataset containing the task-related conversations, and it achieves intent accuracy of 97.7% and slot F1 of 96.8% [10]. Compared with the rulebased keyword extraction, the pre-trained BERT is a contextdependent model that can distinguish the word's meaning in the given context.…”

Section: A Language Servicesmentioning

confidence: 99%

See 1 more Smart Citation

Hey Max, can you help me? An Intuitive Virtual Assistant for Industrial Robots

Li¹,

Chrysostomou²,

Pinto³

et al. 2022

Preprint

Self Cite

1

0

View full text Add to dashboard Cite

Assisting employees in acquiring the knowledge and skills necessary to grasp and use new services and technologies on the shop floor is critical for manufacturers to adapt to Industry 4.0 successfully. In this paper, we employ a Learning, Training, Assistance (LTA) approach and propose a framework for a Language-enabled Virtual Assistant (VA) to facilitate this adaptation. In our system, the human-robot interaction is achieved through spoken natural language and a dashboard implemented as a web-based application. This type of interaction enables operators of all levels to control a collaborative robot intuitively in several industrial scenarios and use it as a complementary tool for developing their competencies. Our proposed framework has been extensively tested with 29 users who completed various tasks while interacting with the proposed VA and industrial robots. Through three different scenarios, we evaluated the usability of the system for LTA based on an established System Usability Scale and the cognitive effort required by the users based on the standardised NASA TLX questionnaire. The qualitative and quantitative results of the study show that users of all levels found the VA user friendly with low requirements for physical and mental effort during the interaction. Additionally, the study demonstrates that the VA enables operators to streamline the learning and training phases of new tasks and improve their user experience during the assistance phase for daily tasks. The source code of the proposed VA and the supplementary material of the user study are accessible at https://bit.ly/VA MAX to support the reproducibility of the proposed framework.

show abstract

“…The last FFNN takes the results of the BERT model as input and outputs a possible user's intent (X intent ) and slots (X slot ). The model is trained and validated on a dialogue dataset containing the task-related conversations, and it achieves intent accuracy of 97.7% and slot F1 of 96.8% [5]. Compared with the rulebased keyword extraction, the pre-trained BERT is a contextdependent model that can distinguish the word's meaning in the given context.…”

Section: A Language Servicesmentioning

confidence: 99%

Hey Max, can you help me? An Intuitive Virtual Assistant for Industrial Robots

Li¹,

Chrysostomou²,

Pinto³

et al. 2022

Preprint

Self Cite

2

0

View full text Add to dashboard Cite

Assisting employees in acquiring the knowledge and skills necessary to grasp and use new services and technologies on the shop floor is critical for manufacturers to adapt to Industry 4.0 successfully. In this paper, we employ a Learning, Training, Assistance (LTA) approach and propose a framework for a Language-enabled Virtual Assistant (VA) to facilitate this adaptation. In our system, the human-robot interaction is achieved through spoken natural language and a dashboard implemented as a web-based application. This type of interaction enables operators of all levels to control a collaborative robot intuitively in several industrial scenarios and use it as a complementary tool for developing their competencies. Our proposed framework has been extensively tested with 29 users who completed various tasks while interacting with the proposed VA and industrial robots. Through three different scenarios, we evaluated the usability of the system for LTA based on an established System Usability Scale and the cognitive effort required by the users based on the standardised NASA TLX questionnaire. The qualitative and quantitative results of the study show that users of all levels found the VA user friendly with low requirements for physical and mental effort during the interaction. Additionally, the study demonstrates that the VA enables operators to streamline the learning and training phases of new tasks and improve their user experience during the assistance phase for daily tasks. The source code of the proposed VA and the supplementary material of the user study are accessible at https://bit.ly/VA MAX to support the reproducibility of the proposed framework.

show abstract

“…In this work, we expand the proof-of-concept we presented previously in [5] where we proposed a virtual assistant (VA) built on a natural language processing architecture used as a digital interpretation of the LTA-FIT model in a smart factory.…”

Section: Introductionmentioning

confidence: 99%

Hey Max, can you help me? An Intuitive Virtual Assistant for Industrial Robots

Li¹,

Chrysostomou²,

Pinto³

et al. 2022

Preprint

Self Cite

0

View full text Add to dashboard Cite

Assisting employees in acquiring the knowledge and skills necessary to grasp and use new services and technologies on the shop floor is critical for manufacturers to adapt to Industry 4.0 successfully. In this paper, we employ a Learning, Training, Assistance (LTA) approach and propose a framework for a Language-enabled Virtual Assistant (VA) to facilitate this adaptation. In our system, the human-robot interaction is achieved through spoken natural language and a dashboard implemented as a web-based application. This type of interaction enables operators of all levels to control a collaborative robot intuitively in several industrial scenarios and use it as a complementary tool for developing their competencies. Our proposed framework has been extensively tested with 29 users who completed various tasks while interacting with the proposed VA and industrial robots. Through three different scenarios, we evaluated the usability of the system for LTA based on an established System Usability Scale and the cognitive effort required by the users based on the standardised NASA-TLX questionnaire. The qualitative and quantitative results of the study show that users of all levels found the VA user friendly with low requirements for physical and mental effort during the interaction. Additionally, the study demonstrates that the VA enables operators to streamline the learning and training phases of new tasks and improve their user experience during the assistance phase for daily tasks. The source code of the proposed VA and the supplementary material of the user study are accessible at https://bit.ly/VA_MAX to support the reproducibility of the proposed framework.

show abstract

Bringing a Natural Language-enabled Virtual Assistant to Industrial Mobile Robots for Learning, Training and Assistance of Manufacturing Tasks

Cited by 16 publications

References 20 publications

Hey Max, can you help me? An Intuitive Virtual Assistant for Industrial Robots

Hey Max, can you help me? An Intuitive Virtual Assistant for Industrial Robots

Hey Max, can you help me? An Intuitive Virtual Assistant for Industrial Robots

Hey Max, can you help me? An Intuitive Virtual Assistant for Industrial Robots

Contact Info

Product

Resources

About