Methodology for stage lighting control based on music emotions

Hsiao, Shih‐Wen; Chen, Shih Kai; Lee, Chu-Hsuan

doi:10.1016/j.ins.2017.05.026

Cited by 21 publications

(12 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our work aims to encourage music providers to categorize music using emotional terms to improve search efficiency and access to music. This work may lead to additional applications, such as music playlist generation based on the listener's heart rate and automatic stage-lighting control based on music emotions [53] [54].…”

Section: Resultsmentioning

confidence: 99%

Emotion Classification System for Digital Music with a Cascaded Technique

2020

View full text Add to dashboard Cite

Music selection is difficult without efficient organization based on metadata or tags, and one effective tag scheme is based on the emotion expressed by the music. However, manual annotation is labor intensive and unstable because the perception of music emotion varies from person to person. This paper presents an emotion classification system for digital music with a resolution of eight emotional classes. Russell’s emotion model was adopted as common ground for emotional annotation. The music information retrieval (MIR) toolbox was employed to extract acoustic features from audio files. The classification system utilized a supervised machine learning technique to recognize acoustic features and create predictive models. Four predictive models were proposed and compared. The models were composed by crossmatching two types of neural networks, i.e., Levenberg-Marquardt (LM) and resilient backpropagation (Rprop), with two types of structures: a traditional multiclass model and the cascaded structure of a binary-class model. The performance of each model was evaluated via the MediaEval Database for Emotional Analysis (DEAM) benchmark. The best result was achieved by the model trained with the cascaded Rprop neural network (accuracy of 89.5%). In addition, correlation coefficient analysis showed that timbre features were the most impactful for prediction. Our work offers an opportunity for a competitive advantage in music classification because only a few music providers currently tag music with emotional terms.

show abstract

Section: Resultsmentioning

confidence: 99%

Emotion Classification System for Digital Music with a Cascaded Technique

2020

View full text Add to dashboard Cite

show abstract

“…The implicit relationship between image and music multimedia data is beyond the semantic dimension in the real world. Color-music [21], lighting-music [22], environmentemotion [23], and image-music [6], [24], [25] can trigger the human experience of synesthesia. The exploration of imagemusic cross-media correlation should change the perspective of general cognition.…”

Section: Cross-media Affective Synesthesia Modelingmentioning

confidence: 99%

“…Hsiao et al explored lighting and music matching mechanism based on emotion analysis for exploring the stage lighting regulations requirements in industry. A support vector regression model was trained to construct the simulation between the two modalities via music emotional strength and genre analysis, and an intelligent lighting control system was developed based on the methodology [22]. Zhao et al proposed an effectiveness method to musicalize images based on their emotions.…”

Section: Cross-media Affective Synesthesia Modelingmentioning

confidence: 99%

See 1 more Smart Citation

Image–Music Synesthesia-Aware Learning Based on Emotional Similarity Recognition

Xing

Zhang

et al. 2019

IEEE Access

View full text Add to dashboard Cite

Synesthesia is a phenomenon in which human experience a cross-sensory interaction in perception. However, it is hard to bridge two sensory modalities in artificial intelligence. Emotion, the universal content across multiple media modalities, can be a cue to connect sensory perceptions for developing computer-based synesthetic intelligence. In this study, we present an image-music, cross-synesthesia-aware model based on their similarity in the emotion space. In this experiment, we built an affective synesthesia database of 250,000 image-music pairs. Multiple music and image features were extracted to form the database. Emotional representation is abstract and complex in perception, and the recognition of emotional similarity is fraught with uncertainty. In this work, Pearson correlation coefficient (PCC) and Euclidean distance (ED) method was compared to obtain the emotional similarity labels of each affective imagemusic pair. The proposed method could predict emotional similarity with mean squared error of 0.0075, demonstrating the effectiveness of our approach and may shed light on the development of cross-modal synesthesia-aware systems. INDEX TERMS Cross-modal retrieval, affective computing, synesthesia-aware modeling, Pearson correlation coefficient.

show abstract

“…Industry-led development in this field is leading to the invention of more sophisticated tools (Laduke 2018) and a greater number of features and options in products. Research in the field includes explorations of alternative paradigms for lighting control, incorporating performer-worn sensors (Saltz 2001), (Francksen et al 2009), spatial positioning (Hakulinen et al 2013), natural language (speech) control (Chang & Canny 2009) augmented and virtual reality interfaces for theatre (Cheok et al 2002), music emotion-based control (Hsiao et al 2017).…”

Section: Developmentmentioning

confidence: 99%

Towards a New Paradigm for Intuitive Theatrical Lighting Control

Schwarz

Cunningham

Smith

2019

Electronic Workshops in Computing

View full text Add to dashboard Cite

A simplified model of a lighting process applied in theatrical productions is one that involves two key players. The first is that of the lighting designer, to produce a set of intentions and plans for the scenes that define the show. The second, the lighting technician, has the job of translating these designs into practice using control equipment, luminaires, and other technical instruments. The lighting design often becomes a 'working document' subject to change and adaptation as the physical reality of the design becomes apparent, and the input of other stakeholders is considered. This process can be a valuable creative tool, and also a difficult technical hurdle to overcome, depending on a varied number of factors. A common frustration with this process is that either the complexity of the task, or difficulty in communication can make it difficult for the final creative vision to be effectively realised. Strains may also arise in the case of small, often touring, theatre companies where the lighting designer and technician may be the same person, and frequently one of the performers as well. Considering the design aspect, there can be challenges in ensuring efficacy of lighting plans between venues in touring productions, with 2D lighting sketches or even 3D computer simulations confined to the paper or screen. From a technical perspective, the role of the lighting technician in theatres and performance situations has included the operation of lighting control equipment during shows. The equipment has evolved over time but has, until recently, been grounded upon the basis of faders and the mixing desk. It is argued that this paradigm has failed to keep pace with the change in other interactive technologies. The ongoing research described in this paper explores existing and upcoming technologies in the field, whilst also seeking to understand the roles and communication workflows of those involved in theatrical lighting to find the best areas to seek improvement, adopting principles of user-centred design. The intention of this research is to develop a new paradigm, and manifestation of it, using a control method for lighting or projection that allows a more intuitive form of operation in theatre productions, which will be scalable and flexible. Lighting design. Pre-visualisation. Collaborative working. User interface design.

show abstract

Methodology for stage lighting control based on music emotions

Cited by 21 publications

References 31 publications

Emotion Classification System for Digital Music with a Cascaded Technique

Emotion Classification System for Digital Music with a Cascaded Technique

Image–Music Synesthesia-Aware Learning Based on Emotional Similarity Recognition

Towards a New Paradigm for Intuitive Theatrical Lighting Control

Contact Info

Product

Resources

About