Developing enhanced conversational agents for social virtual worlds

Abstract

In this paper, we present a methodology for the development of embodied conversational agents for social virtual worlds. The agents provide multimodal communication with their users in which speech interaction is included. Our proposal combines different techniques related to Artificial Intelligence, Natural Language Processing, Affective Computing, and User Modeling. Firstly, the developed conversational agents. A statistical methodology has been developed to model the system conversational behavior, which is learned from an initial corpus and improved with the knowledge acquired from the successive interactions. In addition, the selection of the next system response is adapted considering information stored into user's profiles and also the emotional contents detected in the user's utterances. Our proposal has been evaluated with the successful development of an embodied conversational agent which has been placed in the Second Life social virtual world. The avatar includes the different models and interacts with the users who inhabit the virtual world in order to provide academic information. The experimental results show that the agent's conversational behavior adapts successfully to the specific characteristics of users interacting in such environments.

Robotic Telemedicine for Mental Health: A Multimodal Approach to Improve Human-Robot Engagement

“…This is a common drawback of existing dialogue systems ( Fitzpatrick et al, 2017 ; Harms et al, 2018 ). Nevertheless, great efforts are being made in this promising research field to create natural ‘human-like’ conversations ( Harms et al, 2018 ; Griol et al, 2019 ) 6 , including the exploration of conversational robots and voice-based systems for supporting cognitive impaired individuals ( Cruz-Sandoval et al, 2020 ; Pou-Prom et al, 2020 ; Salichs et al, 2020 ). A possibility for future work is to use the camera of the tablet PC running the robot’s software to automatically recognize user emotions.…”

“…A recent body of work ( Romero et al, 2017 ; Harms et al, 2018 ) has provided novel approaches for the development of conversational agents for increased user engagement. Along similar lines, there has been work on the development of embodied conversational agents—virtual animated characters, usually with the appearance of a human-like avatar, capable of understanding multimodal utterances, such as voice, gestures, and emotion ( Griol et al, 2019 ). These systems aim to provide a more empathetic response based on dialogue and behavior ( Merdivan et al, 2019 ), yet they often cause a sense of discomfort explained by the uncanny valley ( Ciechanowski et al, 2019 ).…”

Los mundos virtuales: una reflexión más allá de lo digital

“…Los participantes de los mundos virtuales tienen la capacidad de interactuar con los demás y con el entorno. La interacción con los demás establece lazos y eventualmente grupos de afiliación (Griol et al 2019).…”

User Modeling and Profiling in Information Systems

“…To avoid this problem, personalized or adaptive systems have been proposed with the aim of presenting the information corresponding to the user's specific needs (e.g., recommender systems, adaptive hypermedia, personalized information retrieval, adaptive user interfaces). A wide range of application domains are concerned (both on the Internet and in enterprise information systems), such as e-commerce (e.g., Amazon) (Smith and Linden 2017;Linden et al 2003), video content (e.g., Youtube, Netflix) (Gomez-Uribe and Hunt 2015, Davidson et al 2010), search engines (e.g., Google) (Speretta and Gauch 2005), e-learning (Wang and Wang 2021;Fink and Kobsa 2002), virtual reality (Griol et al 2019), health (Mao et al 2020;Glykas and Chytas 2004), and tourism (Al Fararni et al 2021;Fink and Kobsa 2002). User modeling or user profiling is very important and fundamental for all these systems and applications because they all require a good inference of the user's needs.…”