A Three-Site Reproduction of the Joint Simon Effect with the NAO Robot

Strait, Megan; Lier, Florian; Bernotat, Jasmin; Wachsmuth, Sven; Eyssel, Friederike Anne; Goldstone, Robert L.; Šabanović, Selma

doi:10.1145/3319502.3374783

Cited by 18 publications

(14 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another replication study was presented by Strait et al (2020) . Three different research teams conducted conceptual replication studies on the joint Simon effect with a robot in three different countries including Germany, United States and Mexico.…”

Section: Confidence In Hri Researchmentioning

confidence: 99%

“…Both, Ullman et al (2021) and Strait et al (2020) emphasized that replications in the field of HRI are particularly difficult because of the large variety of robots that exist, their limited availability, the high costs, and the difficulties in providing run-able source code. Wijnen et al (2020) therefore had a different attempt to replicate another HRI study.…”

Section: Confidence In Hri Researchmentioning

confidence: 99%

“…Replications additionally point out problems in the original work that may have led to the failure of replication (e.g., low power of original work) ( Leichtmann and Nitsch, 2020b ; Ullman et al, 2021 ). However, the three-site replication study by Strait et al (2020) is a first example from recent years that alters the question from “if” to “how” HRI user studies are replicable by suggesting processes of sample size justification, cooperation across different labs, and open materials, code and data ( Strait et al, 2020 ).…”

Section: Confidence In Hri Researchmentioning

confidence: 99%

See 2 more Smart Citations

Crisis Ahead? Why Human-Robot Interaction User Studies May Have Replicability Problems and Directions for Improvement

2022

View full text Add to dashboard Cite

There is a confidence crisis in many scientific disciplines, in particular disciplines researching human behavior, as many effects of original experiments have not been replicated successfully in large-scale replication studies. While human-robot interaction (HRI) is an interdisciplinary research field, the study of human behavior, cognition and emotion in HRI plays also a vital part. Are HRI user studies facing the same problems as other fields and if so, what can be done to overcome them? In this article, we first give a short overview of the replicability crisis in behavioral sciences and its causes. In a second step, we estimate the replicability of HRI user studies mainly 1) by structural comparison of HRI research processes and practices with those of other disciplines with replicability issues, 2) by systematically reviewing meta-analyses of HRI user studies to identify parameters that are known to affect replicability, and 3) by summarizing first replication studies in HRI as direct evidence. Our findings suggest that HRI user studies often exhibit the same problems that caused the replicability crisis in many behavioral sciences, such as small sample sizes, lack of theory, or missing information in reported data. In order to improve the stability of future HRI research, we propose some statistical, methodological and social reforms. This article aims to provide a basis for further discussion and a potential outline for improvements in the field.

show abstract

Section: Confidence In Hri Researchmentioning

confidence: 99%

Section: Confidence In Hri Researchmentioning

confidence: 99%

Section: Confidence In Hri Researchmentioning

confidence: 99%

See 1 more Smart Citation

Crisis Ahead? Why Human-Robot Interaction User Studies May Have Replicability Problems and Directions for Improvement

2022

View full text Add to dashboard Cite

show abstract

“…The data are reported only to help support the idea that we, as HRI researchers, need to better report study metadata for more reproducible and generalizable studies. The goal is to help encourage replicability studies such as "A Three-Site Reproduction of the Joint Simon Effect with the NAO Robo" [4] from HRI'20. With the HRI conference including a reproducbility track in 2020 [3], we hope to see similar focus in other conferences and journals.…”

Section: Discussion and Limitationsmentioning

confidence: 99%

“…Reproducibility has been a relatively prevalent topic for HRI; the International Conference on Human-Robot Interaction itself introduced a "Reproducibility in Human-Robot Interaction" track in 2020, focused on reproducing prior HRI work [3]. HRI researchers have conducted studies to replicate findings from previous work [4], [5]. However, despite these efforts, a lack of reproducibility for studies has been recognized as a "crisis" in psychology and the sciences in general [6], [7].…”

Section: Introductionmentioning

confidence: 99%

What and How Are We Reporting in HRI? A Review and Recommendations for Reporting Recruitment, Compensation, and Gender

Cordero¹,

Groechel²,

Matarić³

2022

Preprint

View full text Add to dashboard Cite

Study reproducibility and generalizability of results to broadly inclusive populations is crucial in any research. Previous meta-analyses in HRI have focused on the consistency of reported information from papers in various categories. However, members of the HRI community have noted that much of the information needed for reproducible and generalizable studies is not found in published papers. We address this issue by surveying the reported study metadata over the past three years (2019 through 2021) of the main proceedings of the International Conference on Human-Robot Interaction (HRI) as well as alt.HRI. Based on the analysis results, we propose a set of recommendations for the HRI community that follow the longer-standing reporting guidelines from human-computer interaction (HCI), psychology, and other fields most related to HRI. Finally, we examine three key areas for user study reproducibility: recruitment details, participant compensation, and participant gender. We find a lack of reporting within each of these study metadata categories: of the 236 studies, 139 studies failed to report recruitment method, 118 studies failed to report compensation, and 62 studies failed to report gender data. This analysis therefore provides guidance about specific types of needed reporting improvements for HRI.

show abstract

Social robot accessories for tailoring and appropriation of social robots

Joshi,

Kamino,

Šabanović

2024

Int J of Soc Robotics

View full text Add to dashboard Cite

The design of robots for everyday use should take into account the specific nature of the individual end-user and the possibility of interactions with multiple users in diverse scenarios, promoting versatility and increasing the chances of their successful adoption in everyday environments. Most robots are designed, however, to perform tasks and interact in typical social scenarios with an abstract human user. We observed a recent surge in the use of accessories with social robots, which aligns with a broader trend of consumers’ preference for personalizing the technologies they interact with. Drawing from the concepts of adaptability and customizability in collaborative systems, we explore the potential use of accessory-like items for social robots to enable low-tech customization and user appropriation, thus enhancing their value and suitability in various social situations. We draw from Human-Computer Interaction and Computer-Supported Co-operative Work literature to show how end-user customizability and appropriation are essential, but less frequently considered, in the study and design of social robots. We conceptualize Social Robot Accessories (SRAs) as a way for end-users to customize robots, and present three studies - (1) a literature survey on accessory-like item use with social robots, (2) a survey of commercially available robot accessories, and (3) a Twitter-based analysis of accessory use for AIBO and LOVOT robots by their users. We use findings from these studies to envision a design space of SRAs for use by Human-Robot Interaction (HRI) researchers.

show abstract

A Three-Site Reproduction of the Joint Simon Effect with the NAO Robot

Cited by 18 publications

References 34 publications

Crisis Ahead? Why Human-Robot Interaction User Studies May Have Replicability Problems and Directions for Improvement

Crisis Ahead? Why Human-Robot Interaction User Studies May Have Replicability Problems and Directions for Improvement

What and How Are We Reporting in HRI? A Review and Recommendations for Reporting Recruitment, Compensation, and Gender

Social robot accessories for tailoring and appropriation of social robots

Contact Info

Product

Resources

About