An Event Driven Approach for Integrating Multi-sensory Effects to Interactive Environments

Santos, Celso A. S.; Neto, Almerindo Nascimento Rehem; Saleme, Estêvão Bissoli

doi:10.1109/smc.2015.178

Cited by 9 publications

(11 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…To cope with the aforementioned challenges and meet those requirements we explore the Play-SEM SER. It is the central component of the PlaySEM platform, a decoupled open source software written in Java to work with multisensory applications 5,6,7,8 . It deals with heterogeneous applications (i.e.…”

Section: Addressing the Challenges With The Use Of The Playsem Sermentioning

confidence: 99%

“…Santos, Neto, and Saleme 6 proposed an approach to integrating event-based multimedia application controlled by gestures with an interactive environment with sensory effects. They argued that the development of an interactive environment is demanding in the sense that it takes time and effort for coping with different software components and information related to sensors and actuators.…”

Section: Integration Of a Gestural Interactive Applicationmentioning

confidence: 99%

“…The integration of a gestural interactive and a distributed mulsemedia systems proposed by Santos, Neto, and Saleme 6 sparked the question of how long it took since the gesture recognition until the delivery of the sensory effects to the users. Saleme and Santos 5 showed evidence that the decoupled approach did not cause relevant delays and it has nearly no synchronization loss with timeline applications such as movies.…”

Section: Temporal Aspects For Mulsemedia Event-based Applicationsmentioning

confidence: 99%

“…The platform was created by Saleme and Santos 5 in order to initially deliver sensory effects such as wind, lighting, and vibration, specified in MPEG-V, to multimedia applications by reusing the PlaySEM Sensory Effects Renderer (SER), the cornerstone component of the PlaySEM architecture. PlaySEM 6,7,8 has evolved to support different profiles of integration following architectural and design patterns to accommodate a multitude of communication and connectivity protocols and sensory effects metadata standard according to the needs of multimedia applications and devices available in the user's environment. We opt for PlaySEM because it has been favorably applied to diversified scenarios.…”

mentioning

confidence: 99%

See 3 more Smart Citations

Coping With the Challenges of Delivering Multiple Sensorial Media

2019

View full text Add to dashboard Cite

Section: Addressing the Challenges With The Use Of The Playsem Sermentioning

confidence: 99%

Section: Integration Of a Gestural Interactive Applicationmentioning

confidence: 99%

Section: Temporal Aspects For Mulsemedia Event-based Applicationsmentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Coping With the Challenges of Delivering Multiple Sensorial Media

2019

View full text Add to dashboard Cite

“…A linguagem NCL ainda não possui suporte à multimodalidade, apesar de propostas anteriores sugerirem tal facilidade ( [2,3]). Porém, o modelo NCM, no qual NCL se baseia, já prevê diversos eventos relacionados com a interação do usuário e também atributos que são alterados com a ocorrência destes eventos.…”

Section: Proposalunclassified

Providing Multimodal User Interaction in NCL

Barreto

Montevecchi²,

Abreu³

et al. 2019

Anais Estendidos Do Simpósio Brasileiro De Sistemas Multimídia E Web (WebMedia)

View full text Add to dashboard Cite

This proposal consists of extending NCL events and connectors to provide multimodal user interaction in NCL, creating new types of events such as touch, motion, eyeMotion, pointer, voiceRecognition, gestureRecognition and faceRecognition. New predefined roles, such as onTouch, onMotion, onEyeMotion, onPointer, on- VoiceRecognition, onFaceRecognition and onFaceRecognition are provided to express connector conditions.

show abstract

A mulsemedia framework for delivering sensory effects to heterogeneous systems

2019

View full text Add to dashboard Cite

Technological advances in computing have allowed multimedia systems to create more immersive experiences for users. Beyond the traditional senses of sight and hearing, researchers have observed that the use of smell, taste, and touch in such systems is becoming increasingly well-received, leading to a new category of multimedia systems called mulsemedia-multiple sensorial media-systems. In parallel, these systems introduce heterogeneous technologies to deliver different sensory effects such as lighting, wind, vibration, and smell, under varied conditions and restrictions. This new paradigm shift poses many challenges, mainly related to mulsemedia integration, delay, responsiveness, sensory effects intensities, wearable and other heterogeneous devices for delivering sensory effects, and remote delivery of mulsemedia components. In addition, new approaches to interacting with multimedia applications have emerged such as multi-touch interfaces, voice processing, and brain-computer interfaces, giving rise to new kinds of complex interactive systems. In this article, we underpin fundamental challenges for delivering multisensory effects to heterogeneous systems. We propose an interoperable mulsemedia framework for coping with these challenges, meeting the emerging requirements. It is achieved through the evolution of an open distributed mulsemedia system. We changed its core following architectural and design patterns to accommodate different profiles of communication, connectivity, and sensory effects metadata standard according to the need of mulsemedia applications and devices available in the user's environment. The results include case studies where the framework has been duly applied.

show abstract

An Event Driven Approach for Integrating Multi-sensory Effects to Interactive Environments

Cited by 9 publications

References 7 publications

Coping With the Challenges of Delivering Multiple Sensorial Media

Coping With the Challenges of Delivering Multiple Sensorial Media

Providing Multimodal User Interaction in NCL

A mulsemedia framework for delivering sensory effects to heterogeneous systems

Contact Info

Product

Resources

About