Conversational Interaction with Multiple Agents Initiated via Proxemics and Gaze

Novick, David G.; Hinojos, Laura J.; Rodriguez, Aaron E.; Camacho, Adriana; Afravi, Mahdokht

doi:10.1145/3284432.3287185

Cited by 6 publications

(3 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…7) Virtual Reality (VR): Not many authors have looked into mixed-initiative interaction in VR. Novick et al [122] made an online mixed-initiative system in VR, where interaction is triggered by proximity and gaze detection. Users reported that they found the interactive mixed-initiative system at least as engaging as a noninteractive narrative scene.…”

Section: Discussionmentioning

confidence: 99%

On Mixed-Initiative Content Creation for Video Games

2022

View full text Add to dashboard Cite

In this article, we present a survey of mixed-initiative methods for the creation of content for video games. We also propose a definition of what mixed initiative implies, as the term lacks a clear specification. The survey includes works not directly aimed at video games but which create content that can potentially be used in games, such as art programs utilizing mixed initiative. Furthermore, we highlight research areas that overlap wholly or partly with mixed initiative, such as casual creators, explainable artificial intelligence, or interactive evolutionary computation. We examine these and several other topics in the context of mixed initiative. Finally, we provide a catalogue of typical techniques and challenges connected with mixed initiative before considering future directions.

show abstract

Section: Discussionmentioning

confidence: 99%

On Mixed-Initiative Content Creation for Video Games

2022

View full text Add to dashboard Cite

show abstract

“…Such social factors as emotional expressions (Ruggiero et al, 2017;Bönsch et al, 2018Bönsch et al, , 2020, motion attractiveness (Zibrek et al, 2020), gaze (Bailenson et al, 2001(Bailenson et al, , 2003, and coughing behaviors (Shiomi et al, 2022) affect proxemics in human-agent interactions as well as in human-human interactions in physical environments. Based on the foundation of proxemics, various works have proposed designs for virtual and robotic behaviors (Althaus et al, 2004;Kirby et al, 2009;Svenstrup et al, 2010;Novick et al, 2018). These works show how proxemics also affects human-agent and human-human interaction, and knowledge of proxemics is critical for designing acceptable pre-interaction behaviors of agents.…”

Section: Proxemics In Vr Environmentsmentioning

confidence: 99%

Effects of appearance and gender on pre-touch proxemics in virtual reality

et al. 2023

View full text Add to dashboard Cite

Virtual reality (VR) environments are increasingly popular for various applications, and the appearance of virtual characters is a critical factor that influences user behaviors. In this study, we aimed to investigate the impact of avatar and agent appearances on pre-touch proxemics in VR. To achieve this goal, we designed experiments utilizing three user avatars (man/woman/robot) and three virtual agents (man/woman/robot). Specifically, we measured the pre-touch reaction distances to the face and body, which are the distances at which a person starts to feel uncomfortable before being touched. We examined how these distances varied based on the appearances of avatars, agents, and user gender. Our results revealed that the appearance of avatars and agents significantly impacted pre-touch reaction distances. Specifically, those using a female avatar tended to maintain larger distances before their face and body to be touched, and people also preferred greater distances before being touched by a robot agent. Interestingly, we observed no effects of user gender on pre-touch reaction distances. These findings have implications for the design and implementation of VR systems, as they suggest that avatar and agent appearances play a significant role in shaping users’ perceptions of pre-touch proxemics. Our study highlights the importance of considering these factors when creating immersive and socially acceptable VR experiences.

show abstract

“…Additionally, a given agent may be more proficient at a specific domain over another i.e A finance CA is better suited to answer finance questions. As a result, users are taxed with the burden of learning and adopting multiple agents leading to an increase in the cognitive load of interacting with agents, further discouraging the proliferation of their usage (Dubiel et al, 2020;Novick et al, 2018;Saltsman et al, 2019). This is escalated further as the number of conversational agents deployed into the market continues to increase.…”

Section: Introductionmentioning

confidence: 99%

One Agent To Rule Them All: Towards Multi-agent Conversational AI

Clarke¹,

Peper²,

Krishnamurthy³

et al. 2022

Preprint

View full text Add to dashboard Cite

The increasing volume of commercially available conversational agents (CAs) on the market has resulted in users being burdened with learning and adopting multiple agents to accomplish their tasks. Though prior work has explored supporting a multitude of domains within the design of a single agent, the interaction experience suffers due to the large action space of desired capabilities. To address these problems, we introduce a new task BBAI: Black-Box Agent Integration, focusing on combining the capabilities of multiple black-box CAs at scale. We explore two techniques: question agent pairing and question response pairing aimed at resolving this task. Leveraging these techniques, we design One For All (OFA), a scalable system that provides a unified interface to interact with multiple CAs. Additionally, we introduce MARS: Multi Agent Response Selection, a new encoder model for question response pairing that jointly encodes user question and agent response pairs. We demonstrate that OFA is able to automatically and accurately integrate an ensemble of commercially available CAs spanning disparate domains. Specifically, using the MARS encoder we achieve the highest accuracy on our BBAI task, outperforming strong baselines.

show abstract

Conversational Interaction with Multiple Agents Initiated via Proxemics and Gaze

Cited by 6 publications

References 6 publications

On Mixed-Initiative Content Creation for Video Games

On Mixed-Initiative Content Creation for Video Games

Effects of appearance and gender on pre-touch proxemics in virtual reality

One Agent To Rule Them All: Towards Multi-agent Conversational AI

Contact Info

Product

Resources

About