From Human to Robot Everyday Activity

Mason, Celeste; Gadzicki, Konrad; Meier, Moritz; Ahrens, Florian; Kluß, Thorsten; Maldonado, Jaime; Putze, Felix; Fehr, Thorsten; Zetzsche, Christoph; Herrmann, Manfred; Schill, Kerstin; Schultz, Tanja

doi:10.1109/iros45743.2020.9340706

Cited by 10 publications

(4 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This study investigates EEG responses to distractions and hesitations in human-robot interaction. It integrates previous findings from the MAS Lab with the work of CSL and colleagues within the DFG CRC 1320 Everyday Activity Science and Engineering (EASE), where we provide unique and critical contextual background for robots based on the recording, processing, modeling, and interpretation of human activities, perceptions, and feedback [26]. For this purpose, biosignals resulting from the activity of the brain, the muscles, and the eyes, which are correlated to motion, communication, and other mental tasks, are recorded and interpreted to provide insight into diverse aspects of human behavior that enable them to masterfully perform everyday activities with little effort or attention [24].…”

Section: State Of the Artmentioning

confidence: 88%

“…The following study would not have been conducted without LabLinking, since it builds on the complementary expertise and equipment of two laboratories: the Medical Assistance Systems Group (MAS) at Bielefeld University with its rich expertise in social robotics based on robots such as Pepper, Nao, or Flobi [19][20][21], and the Cognitive Systems Lab (CSL) at University of Bremen with vast experience in biosignal-adaptive cognitive systems [22] based on multimodal biosignal acquisition [23] and processing using machine learning methods [24], including the recording and interpretation of spoken communication [25] and high-density EEG in the context of intelligent robots and systems [26].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

EEG Correlates of Distractions and Hesitations in Human–Robot Interaction: A LabLinking Pilot Study

Richter

Putze

Ivucic

et al. 2023

MTI

Self Cite

View full text Add to dashboard Cite

In this paper, we investigate the effect of distractions and hesitations as a scaffolding strategy. Recent research points to the potential beneficial effects of a speaker’s hesitations on the listeners’ comprehension of utterances, although results from studies on this issue indicate that humans do not make strategic use of them. The role of hesitations and their communicative function in human-human interaction is a much-discussed topic in current research. To better understand the underlying cognitive processes, we developed a human–robot interaction (HRI) setup that allows the measurement of the electroencephalogram (EEG) signals of a human participant while interacting with a robot. We thereby address the research question of whether we find effects on single-trial EEG based on the distraction and the corresponding robot’s hesitation scaffolding strategy. To carry out the experiments, we leverage our LabLinking method, which enables interdisciplinary joint research between remote labs. This study could not have been conducted without LabLinking, as the two involved labs needed to combine their individual expertise and equipment to achieve the goal together. The results of our study indicate that the EEG correlates in the distracted condition are different from the baseline condition without distractions. Furthermore, we could differentiate the EEG correlates of distraction with and without a hesitation scaffolding strategy. This proof-of-concept study shows that LabLinking makes it possible to conduct collaborative HRI studies in remote laboratories and lays the first foundation for more in-depth research into robotic scaffolding strategies.

show abstract

Section: State Of the Artmentioning

confidence: 88%

Section: Introductionmentioning

confidence: 99%

EEG Correlates of Distractions and Hesitations in Human–Robot Interaction: A LabLinking Pilot Study

Richter

Putze

Ivucic

et al. 2023

MTI

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, annotating naturalistic expressions and their contexts can be more complicated and labor-intensive than laboratory datasets. Advanced multimodal annotation tools [ 88 ] may help provide multimodal annotation to evaluate facial expressions together with other nonverbal modalities and rich contextual information to provide accurate portrayals of the interaction between facial expressions and contexts. In the following section, we discuss the novel applications of MLLMs that could circumvent the need for extensively annotated datasets, fostering further advancement in naturalistic affective research.…”

Section: Analyzing Naturalistic Facial Expressions With Deep Learningmentioning

confidence: 99%

Understanding Naturalistic Facial Expressions with Deep Learning and Multimodal Large Language Models

Bian,

Küster,

Liu

et al. 2023

Sensors

View full text Add to dashboard Cite

This paper provides a comprehensive overview of affective computing systems for facial expression recognition (FER) research in naturalistic contexts. The first section presents an updated account of user-friendly FER toolboxes incorporating state-of-the-art deep learning models and elaborates on their neural architectures, datasets, and performances across domains. These sophisticated FER toolboxes can robustly address a variety of challenges encountered in the wild such as variations in illumination and head pose, which may otherwise impact recognition accuracy. The second section of this paper discusses multimodal large language models (MLLMs) and their potential applications in affective science. MLLMs exhibit human-level capabilities for FER and enable the quantification of various contextual variables to provide context-aware emotion inferences. These advancements have the potential to revolutionize current methodological approaches for studying the contextual influences on emotions, leading to the development of contextualized emotion models.

show abstract

“…In particular, we describe the benefits of the simultaneous recordings of biosignal data corresponding to everyday activities. These recordings leverage several modalities captured by various sensors, ranging from wearable low-cost acceleration sensors to high-dimensional costly functional brain imaging devices [7].…”

Section: Lablinking Ecosystemsmentioning

confidence: 99%

Linking Labs: Interconnecting Experimental Environments

Schultz¹,

Putze²,

Fehr³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We introduce the concept of LabLinking: a technology-based interconnection of experimental laboratories across institutions, disciplines, cultures, languages, and time zones -in other words experiments without borders. In particular, we introduce LabLinking levels (LLL), which define the degree of tightness of empirical interconnection between labs. We describe the technological infrastructure in terms of hard-and software required for the respective LLLs and present examples of linked laboratories along with insights about the challenges and benefits. In sum, we argue that linked labs provide a unique platform for a continuous exchange between scientists and experimenters, thereby enabling a time synchronous execution of experiments performed with and by decentralized user and researchers, improving outreach and ease of subject recruitment, allowing to establish new experimental designs and to incorporate a panoply of complementary biosensors, devices, hard-and software solutions.

show abstract

From Human to Robot Everyday Activity

Cited by 10 publications

References 37 publications

EEG Correlates of Distractions and Hesitations in Human–Robot Interaction: A LabLinking Pilot Study

EEG Correlates of Distractions and Hesitations in Human–Robot Interaction: A LabLinking Pilot Study

Understanding Naturalistic Facial Expressions with Deep Learning and Multimodal Large Language Models

Linking Labs: Interconnecting Experimental Environments

Contact Info

Product

Resources

About