Towards a reference ontology on mulsemedia systems

Abstract

The use of multiple senses in interactive applications has become increasingly feasible due to the upsurge of commercial, off-the-shelf devices to produce sensory effects. Creating Multiple Sensorial Media (MulSeMedia) immersive systems requires understanding their digital ecosystem. Mulsemedia systems encompass a set of applications, and devices of different types assembled to communicate or express feelings from the virtual world to the real world. Despite existing standards, tools, and recent research devoted to them, there is still a lack of formal and explicit representation of what mulsemedia is. Misconceptions could eventually lead to the construction of solutions that might not take into account reuse, integration, standardization, among other design features. In this paper, we propose to establish a common conceptualization about mulsemedia systems through a reference ontology, named MulseOnto, covering their main notions. To evaluate it, we applied ontology verification and validation techniques, including assessment by humans and a data-driven approach. The results showed that MulseOnto can be used as a consensual conceptual model for exploring the knowledge about the whole chain of mulsemedia systems. CCS CONCEPTS • General and reference → Reference works; • Information systems → Multimedia information systems; • Human-centered computing → Ubiquitous and mobile computing theory, concepts and paradigms;

QoE of cross-modally mapped Mulsemedia: an assessment using eye gaze and heart rate

“…camera, microphone, motion capture) that capture real-world information, or synthesized using computers (e.g. a virtual 3D space in a game) [56]. Many tools have been developed to aid this process, such as SEVino [75], SMURF [32], RoSE Studio [5], and Real 4D studio [59].…”

“…Mulsemedia players and renderers to be used with other multimedia applications have also been created to reproduce and deliver mulsemedia experiences, notably SEMP [75] and PlaySEM [55], which are open-source. A mulsemedia system entails weaving multiple technologies to connect different entities, distribute the sensory signals, and render sensory effects appropriately Saleme et al [56]. Whilst developing mulsemedia systems, it is crucial to have ways to deliver different sensory content consistently as well as of paramount importance to be aware of the challenges that might arise when delivering mulsemedia [57].…”

A mulsemedia framework for delivering sensory effects to heterogeneous systems

“…A distinction between multimedia and mulsemedia is made in [48]. Whilst the former aims at stimulating sight and hearing, the latter combines the former plus at least one more sense among touch, smell, and taste.…”

A Conceptual Architecture and a Framework for Dealing with Variability in Mulsemedia Systems

“…The increasing interest in digital immersive experiences has drawn the attention of researchers into understanding human perception whilst adding sensory effects to multimedia systems such as Virtual Reality (VR) and Augmented Reality (AR) applications, content players, and games. These so-called mulsemedia-multiple sensorial mediasystems [Saleme et al 2018a, Saleme et al 2019c] are capable of delivering wind, smell, vibration, among others, along with audiovisual content to enhance users' Quality of Experience (QoE) and to improve immersiveness in applied areas such as entertainment, healthcare, education, culture, and marketing.…”