An Adaptive Human Sensor Framework for Human-Robot Collaboration

Abstract

Manufacturing challenges are increasing the demands for more agile and dexterous means of production. At the same time, these systems aim to maintain or even increase productivity. The challenges risen from these developments can be tackled through Human-Robot Collaboration (HRC). HRC requires effective task distribution according to each parties’ distinctive strengths, which is envisioned to generate synergetic effects. To enable a seamless collaboration, the human and robot require a mutual awareness, which is challenging, due to the human and robot “speaking” different languages as in analogue and digital. Thus, this challenge can be addressed by equipping the robot with a model of the human. Despite a range of models being available, data-driven models of the human are still at an early stage. This paper proposes an adaptive human sensor framework, which incorporates objective, subjective, and physiological metrics, as well as associated Machine Learning. Thus, it is envisioned to adapt to the uniqueness and dynamic nature of human behavior. To test the framework, a validation experiment was performed, including 18 participants, which aims to predict Perceived Workload during two scenarios, namely a manual and an HRC assembly task. Perceived Workloads are described to have a substantial impact on a human operator’s task performance. Throughout the experiment physiological data from an electroencephalogram (EEG), an electrocardiogram (ECG), and respiration sensor was collected and interpreted. For subjective metrics, the standardized NASA Task Load Index was used. Objective metrics included task completion time and number of errors/assistance requests. Overall, the framework revealed a promising potential towards an adaptive behavior, which is ultimately envisioned to enable a more effective HRC.

Coordinating human-robot collaboration by EEG-based human intention prediction and vigilance control

“…In order to predict the human's intention earlier, EEGbased methods have been investigated in previous studies (Shafiul Hasan et al, 2020). For example, the Bereitschaftspotential, a conspicuous deflection in the EEG signal about 500 ms before movement onset, has been used to predict when the human will move the arms and which kind of grasp action will be executed (Buerkle et al, 2021;Xu et al, 2021).…”

“…The time point of a movement onset can be predicted from the readiness potential, a conspicuous deflection in the EEG signal which starts about 500 ms before the movement. This signal can be used to predict whether the human will move the left or right arm, and the speed of the robot can be adjusted according to whether or not the human is in the shared workspace (Cooper et al, 2020;Buerkle et al, 2021). Movement-related cortical potentials, recorded by an EEG amplifier around the time of movement onset, have been shown to be informative about the upcoming grasp action, e.g., palmar, pinch, etc.…”

“…Most studies on BCI methods for HRC focus on the development of the BCI component (Chen et al, 2018(Chen et al, , 2019 or evaluate the interaction in simulation (Buerkle et al, 2021). Here, we demonstrate methods for integrating BCI output as well as human arm tracking data with the robot controller in a physical setup.…”

Diseño de una arquitectura heterárquica de control de la producción centrada en el humano

“…Como ejemplo en Buerkle et al (2021), para analizar los datos de los sensores se usa una Arquitectura Unificada OPC, ya que es una arquitectura orientada a servicios independiente de la plataforma, que permite agregar o quitar sensores cuando sea necesario. Igualmente, el uso de diversas herramientas para realizar Big Data permitirá comunicar información concreta según las necesidades establecidas.…”

“…La desventaja de una métrica subjetiva es que los datos se recopilan en retrospectiva, en la que se espera que los participantes recuerden, interpreten y expliquen experiencias, que son susceptibles de sesgos cognitivos. Sin embargo, las métricas subjetivas estandarizadas como la NASA-TLX tienen como objetivo eliminar este sesgo al máximo (Buerkle et al, 2021). Su validez, comprobada en numerosas investigaciones, y su facilidad de uso hacen que sea el instrumento de evaluación de carga mental más ampliamente utilizado (Díaz Ramiro et al, 2010).…”