Multimodal object oriented user interfaces in mobile affective interaction

Alepis, Efthimios; Virvou, Maria

doi:10.1007/s11042-011-0744-y

Cited by 8 publications

(4 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similarly, speech analysis can be used to identify emotions based on the tone, pitch, and other features of a person's voice [70]. Multi-modal affect recognition can combine many sources of evidence from multiple modalities, such as the keyboard, the microphone, the camera and other sources [71][72][73]. Emotion generation is the process of using artificial intelligence (AI) algorithms to create computer feedback that conveys empathy to the users.…”

Section: Artificial Intelligence and User Modelling In Service Of Use...mentioning

confidence: 99%

Artificial Intelligence and User Experience in reciprocity: Contributions and state of the art

Virvou¹

2023

IDT

Self Cite

View full text Add to dashboard Cite

Among the primary aims of Artificial Intelligence (AI) is the enhancement of User Experience (UX) by providing deep understanding, profound empathy, tailored assistance, useful recommendations, and natural communication with human interactants while they are achieving their goals through computer use. To this end, AI is used in varying techniques to automate sophisticated functions in UX and thereby changing what UX is apprehended by the users. This is achieved through the development of intelligent interactive systems such as virtual assistants, recommender systems, and intelligent tutoring systems. The changes are well received, as technological achievements but create new challenges of trust, explainability and usability to humans, which in turn need to be amended by further advancements of AI in reciprocity. AI can be utilised to enhance the UX of a system while the quality of the UX can influence the effectiveness of AI. The state of the art in AI for UX is constantly evolving, with a growing focus on designing transparent, explainable, and fair AI systems that prioritise user control and autonomy, protect user data privacy and security, and promote diversity and inclusivity in the design process. Staying up to date with the latest advancements and best practices in this field is crucial. This paper conducts a critical analysis of published academic works and research studies related to AI and UX, exploring their interrelationship and the cause-effect cycle between the two. Ultimately, best practices for achieving a successful interrelationship of AI in UX are identified and listed based on established methods or techniques that have been proven to be effective in previous research reviewed.

show abstract

Section: Artificial Intelligence and User Modelling In Service Of Use...mentioning

confidence: 99%

Artificial Intelligence and User Experience in reciprocity: Contributions and state of the art

Virvou¹

2023

IDT

Self Cite

View full text Add to dashboard Cite

show abstract

“…As we mentioned above, the concept of multimodality has been around for quite a long time now and not just in theory. Multimodal systems have been deployed and tested in fields such as education [1,13,20,27,29,32,68], healthcare [6,7], marketing [5,30,38,42,58] or gaming [22,59].…”

Section: Multimodal Emotion Recognition Systemsmentioning

confidence: 99%

“…Alepis and Virbou [1] developed a proposal which is the closest to our proposal we have been able to find, although it is designed specifically for mobile phones and handheld devices.…”

Section: Multimodal Emotion Recognition Systemsmentioning

confidence: 99%

Building a three-level multimodal emotion recognition framework

García-García¹,

Lozano

Penichet

et al. 2022

Multimed Tools Appl

View full text Add to dashboard Cite

Multimodal emotion detection has been one of the main lines of research in the field of Affective Computing (AC) in recent years. Multimodal detectors aggregate information coming from different channels or modalities to determine what emotion users are expressing with a higher degree of accuracy. However, despite the benefits offered by this kind of detectors, their presence in real implementations is still scarce for various reasons. In this paper, we propose a technology-agnostic framework, HERA, to facilitate the creation of multimodal emotion detectors, offering a tool characterized by its modularity and the interface-based programming approach adopted in its development. HERA (Heterogeneous Emotional Results Aggregator) offers an architecture to integrate different emotion detection services and aggregate their heterogeneous results to produce a final result using a common format. This proposal constitutes a step forward in the development of multimodal detectors, providing an architecture to manage different detectors and fuse the results produced by them in a sensible way. We assessed the validity of the proposal by testing the system with several developers with no previous knowledge about affective technology and emotion detection. The assessment was performed applying the Computer System Usability Questionnaire and the Twelve Cognitive Dimensions Questionnaire, used by The Visual Studio Usability group at Microsoft, obtaining positive results and important feedback for future versions of the system.

show abstract

“…It is not surprising then that conventional uni-modal human-machine interactions lag in performance, robustness and naturalness when compared with human-human interactions. Recently, there has been increasing research interest in jointly processing information in multiple modalities and mimicking human-human multimodal interactions [2,4,5,9,13,14,16,18,19,21,22]. For example, human speech production and perception are bimodal in nature: visual cues have a broad influence on perceived auditory stimuli [17].…”

mentioning

confidence: 99%

Multimodal joint information processing in human machine interaction: recent advances

Xie

Deng

Cox

2013

Multimed Tools Appl

View full text Add to dashboard Cite

Interaction between humans is a multimodal process in nature, which integrates the different modalities of vision, hearing, gesture and touch. It is not surprising then that conventional uni-modal human-machine interactions lag in performance, robustness and naturalness when compared with human-human interactions. Recently, there has been increasing research interest in jointly processing information in multiple modalities and mimicking human-human multimodal interactions [2,4,5,9,13,14,16,18,19,21,22]. For example, human speech production and perception are bimodal in nature: visual cues have a broad influence on perceived auditory stimuli [17]. The latest research in speech processing has shown that integration of both auditory and visual information, i.e., acoustic speech combined with facial and lip motions, achieves significant performance improvement in tasks such as speech recognition [3] and emotion recognition [16]. However, information from difference modalities can be difficult to integrate: the representations of different signals are heterogeneous and the signals are often asynchronous and loosely-coupled. Therefore, finding more effective ways to integrate and jointly process information from different modalities is essential for the success of such multimodal human-machine interactive systems. Meanwhile, multimodal applications are becoming increasingly important, especially for mobile computing and digital entertainment scenarios, such as natural user interfaces (NUI) on smartphones and motion sensing gaming.This special issue aims to bring together work by researchers and technologists engaged in the development of multimodal technologies for information processing, emerging multimedia applications and user-centric human computer interaction. We received more than L. Xie ( )

show abstract

Multimodal object oriented user interfaces in mobile affective interaction

Cited by 8 publications

References 31 publications

Artificial Intelligence and User Experience in reciprocity: Contributions and state of the art

Artificial Intelligence and User Experience in reciprocity: Contributions and state of the art

Building a three-level multimodal emotion recognition framework

Multimodal joint information processing in human machine interaction: recent advances

Contact Info

Product

Resources

About