Streaming Media with Sensory Effect

Choi, Bumsuk; Lee, Eun-Seo; Yoon, Kyoungro

doi:10.1109/icisa.2011.5772390

Cited by 19 publications

(11 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…a virtual 3D space in a game) [56]. Many tools have been developed to aid this process, such as SEVino [75], SMURF [32], RoSE Studio [5], and Real 4D studio [59]. The works of Kim et al [33] and Oh and Huh [48] are endeavors to automatically produce mulsemedia metadata.…”

Section: Mulsemedia and Qoementioning

confidence: 99%

See 1 more Smart Citation

QoE of cross-modally mapped Mulsemedia: an assessment using eye gaze and heart rate

Mesfin

Hussain

Kani‐Zabihi

et al. 2020

Multimed Tools Appl

View full text Add to dashboard Cite

A great deal of research effort has been put in exploring crossmodal correspondences in the field of cognitive science which refer to the systematic associations frequently made between different sensory modalities (e.g. high pitch is matched with angular shapes). However, the possibilities cross-modality opens in the digital world have been relatively unexplored. Therefore, we consider that studying the plasticity and the effects of crossmodal correspondences in a mulsemedia setup can bring novel insights about improving the human-computer dialogue and experience. Mulsemedia refers to the combination of three or more senses to create immersive experiences. In our experiments, users were shown six video clips associated with certain visual features based on color, brightness, and shape. We examined if the pairing with crossmodal matching sound and the corresponding auto-generated haptic effect, and smell would lead to an enhanced user QoE. For this, we used an eye-tracking device as well as a heart rate monitor wristband to capture users' eye gaze and heart rate whilst they were experiencing mulsemedia. After each video clip, we asked the users to complete an on-screen questionnaire with a set of questions related to smell, sound and haptic effects targeting their enjoyment and perception of the experiment. Accordingly, the eye gaze and heart rate results showed significant influence of the cross-modally mapped multisensorial effects on the users' QoE. Our results highlight that when the olfactory content is crossmodally congruent with the visual content, the visual attention of the users seems shifted towards the correspondent visual feature. Crosmodally matched media is also shown to result in an enhanced QoE compared to a video only condition.

show abstract

Section: Mulsemedia and Qoementioning

confidence: 99%

“…The olfactory emitting device was provided by the Exhalia SBi4, 5 which was considered by previous research more reliable and more robust than existing devices [45]. This was placed at 0.5 m from the assessor, allowing her/him to detect the smell in 2.7-3.2 s, as shown in [44].…”

Section: Experimental Apparatusmentioning

confidence: 99%

QoE of cross-modally mapped Mulsemedia: an assessment using eye gaze and heart rate

Mesfin

Hussain

Kani‐Zabihi

et al. 2020

Multimed Tools Appl

View full text Add to dashboard Cite

show abstract

“…Mulsemedia authoring tools have been developed for almost a decade. SEVino [37], SMURF [21], RoSE Studio [7], and Real 4D studio [33] are examples of tools that support authors in adding sensory effects, usually represented as MPEG-V metadata, to AV contents. Players compatible with MPEG-V such as Sensorama [6], SEMP [37], Sensible Media Simulator [21], Multimedia Multisensorial 4D Platform [4], and PlaySEM [31] are able to reproduce this kind of authored content.…”

Section: Related Workmentioning

confidence: 99%

“…The process of producing interoperable mulsemedia metadata involves making an MPEG-V compatible XML file that contains entities describing different sensory effect presentation features (beginning and the end of each effect time span, its intensity, fading and so forth). This can be done with the help of an authoring tool, which enhances considerably the efficiency of the process overall [37,21,7,33]. This tool allows authors to abstract the difficulty of writing an XML application, through an intuitive graphical interface whereby they can pick up a movie scene, see what they would feel whilst immersed in the scene, and then, arrange the sensory effects on it.…”

Section: Introductionmentioning

confidence: 99%

Crowdsourcing authoring of sensory effects on videos

Amorim

Saleme

Neto

et al. 2019

Multimed Tools Appl

View full text Add to dashboard Cite

Human perception is inherently multi-sensorial involving five traditional senses: sight, hearing, touch, taste, and smell. In contrast to traditional multimedia, based on audio and visual stimuli, mulsemedia seek to stimulate all the human senses. One way to produce multi-sensorial content is authoring videos with sensory effects. These effects are represented as metadata attached to the video content, which are processed and rendered through physical devices into the user's environment. However, creating sensory effects metadata is not a trivial activity because authors have to identify carefully different details in a scene such as the exact point where each effect starts, finishes, and also its presentation features such as intensity, direction, etc. It is a subjective task that requires accurate human perception and time. In this article, we aim at finding out whether a crowdsourcing approach is suitable for authoring coherent sensory effects associated with video content. Our belief is that the combination of a collective common sense to indicate time intervals of sensory effects with an expert fine-tuning is a viable way to generate sensory effects from the point of view of users. To carry out the experiment, we selected three videos from a public mulsemedia dataset, sent them to the crowd through a cascading microtask approach. The results showed that the crowd can indicate intervals in which users agree that there should be insertions of sensory effects, revealing a way of sharing authoring between the author and the crowd.

show abstract

“…It also creates the sense of buoyancy, water pressure, water temperature, and resistance changes sensory effects in accordance with changes in the water depth of the 3D underwater environment. Here, the sensory effect information in the virtual underwater environment is redefined based on the structure of metadata (SEM: Sensory Effect Metadata) defined in standard technologies MPEG-V [16], related to sensory effects. These data are formed with standardized data (XML instances), changed into device commands by a sensory effects manager, and then reproduced as sensory effects via devices in the real world.…”

Section: Multisensory Effect Reproduction Systemmentioning

confidence: 99%

Design and Assessment of a Virtual Underwater Multisensory Effects Reproducing Simulation System

Kim

Youn

Ji³

et al. 2014

International Journal of Distributed Sensor Networks

View full text Add to dashboard Cite

With current developments in virtual reality technology and advances in the marine industry, various virtual underwater simulation systems that help users to indirectly experience the marine environment via a variety of virtual contents are being developed. In general, underwater simulation systems that use virtual reality developed using three-dimensional graphics technology. Therefore, they are limited in terms of their ability to imbue users with a sense of immersion and presence. In addition, the expansion of the sensory range, with which users can actually experience various underwater environmental phenomena while they are in underwater simulations, is still insufficient. This paper proposes an immersive multisensory effect reproduction system that provides an improved sense of underwater reality for users. The results of comparative user evaluation tests conducted regarding the sense of reality imbued with our proposed system indicate that its sensory presence effect is superior to that of conventional systems.

show abstract

Streaming Media with Sensory Effect

Cited by 19 publications

References 0 publications

QoE of cross-modally mapped Mulsemedia: an assessment using eye gaze and heart rate

QoE of cross-modally mapped Mulsemedia: an assessment using eye gaze and heart rate

Crowdsourcing authoring of sensory effects on videos

Design and Assessment of a Virtual Underwater Multisensory Effects Reproducing Simulation System

Contact Info

Product

Resources

About