Multimodal and mobile conversational Health and Fitness Companions

Abstract

Multimodal conversational spoken dialogues using physical and virtual agents provide a potential interface to motivate and support users in the domain of health and fitness. In this paper we present how such multimodal conversational Companions can be implemented to support their owners in various pervasive and mobile settings. We present concrete system architectures, virtual, physical and mobile multimodal interfaces, and interaction management techniques for such companions. In particular, we present how knowledge representation and separation of low-level interaction modelling from high-level reasoning at the domain level makes it possible to implement distributed, but still coherent, interaction with Companions. The distribution is enabled by using a dialogue plan to communicate information from domain level planner to dialogue management and from there to a separate mobile interface. The model enables each part of the system to handle the same information from its own perspective without containing overlapping * Corresponding author Email addresses: mturunen@cs.uta.fi (Markku Turunen), jh@cs.uta.fi (Jaakko Hakulinen), olovs@sics.se (Olov Ståhl), gamback@sics.se (Björn Gambäck), preben@sics.se (Preben Hansen), mcrg@tid.es (Mari C. Rodríguez Gancedo), e.rsai@tid.es (Raúl Santos de la Cámara), c.g.smith@tees.ac.uk 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 logic, and makes it possible to separate task-specific and conversational dialogue management from each other. In addition to technical descriptions, we present results from the first evaluations of the Companions interfaces.

ARTIFICIAL CONVERSATIONAL COMPANIONS - A Requirements Analysis

“…Recent contributions in the domain of ACC are the EU-Companions project 1 with the implementation of a "How Was Your Day" Companion (HWYDCompanion) (Pulman et al, 2010) for talking about job-related topics with the user, the Senior Companion (SC) for reminiscing about images (Wilks et al, 2011), and the Health and Fitness Companion (HFC) for daily exercise planning, leisure activities and diet (Turunen et al, 2011). T.Bickmore's precompanion work on Relational Agents with the implementation of a personal coach Laura (Bickmore, 2003) focuses on social-emotional relationships between humans and computer agents.…”

Empathy and Mobile Applications

“…Detection of the user's current physical position will be performed by a Global Positioning System [17]. Ståhl et al (2011) presented The Health and Fitness Companion (HFC) acts as a conversational partner whose overall aim is to build a long-term relationship with the user. Planning the day becomes a compromise between the user and the system.…”

“…However, the character acts more as a companion than a personal trainer, since it takes a persistent role in the user's daily life. The Health and Fitness Companion is an a!ective, empathic agent aims to set up a long lasting social and emotional relationship so as the user support overall lifestyle changes in his/her daily habits [18], [19].…”

Smart tailoring of real-time physical activity coaching systems

“…As such, the authors do not provide any background regarding theories of behavior change or tailoring aspects (goal setting and context awareness) employed in the system. Details on the evaluation strategies can be found in (Benyon et al 2008) and further details including evaluation results are presented in (Turunen et al 2011). …”

“…The idea is that having the same "virtual coach agent" as representation on both devices will support the notion that the different devices are merely different output platform for the same virtual coach. This notion of having the same virtual coach presentation in multiple interfaces to accommodate persistence is one of the main features of the Mobile Fitness Companion described in (Ståhl et al 2008, Ståhl et al 2009, Turunen et al 2011.…”