Music Emotion Recognition With Audio and Lyrics Features

Nanayakkara, Charini; Caldera, H. A.

doi:10.17781/p002150

Cited by 2 publications

(2 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The preprocess such: clear the symbol or special characters, inlining the word, lowercase the capital word, and annotating the lyrics with positive, neutral, and negative. The next thing to do is list the artist, title, annotation, and lyrics in the comma separated values (CSV) file [26]. Afterward, the CSV files were  ISSN: 2302-9285 uploaded to Google drive to ease the experiments.…”

Section: Xlnet Transformersmentioning

confidence: 99%

Multimodal music emotion recognition in Indonesian songs based on CNN-LSTM, XLNet transformers

Sams

Zahra²

2023

Bulletin EEI

View full text Add to dashboard Cite

Music carries emotional information and allows the listener to feel the emotions contained in the music. This study proposes a multimodal music emotion recognition (MER) system using Indonesian song and lyrics data. In the proposed multimodal system, the audio data will use the mel spectrogram feature, and the lyrics feature will be extracted by going through the tokenizing process from XLNet. Convolutional long short term memory network (CNN-LSTM) performs the audio classification task, while XLNet transformers performs the lyrics classification task. The outputs of the two classification tasks are probability weight and actual prediction with the value of positive, neutral, and negative emotions, which are then combined using the stacking ensemble method. The combined output will be trained into an artificial neural network (ANN) model to get the best probability weight output. The multimodal system achieves the best performance with an accuracy of 80.56%. The results showed that the multimodal method of recognizing musical emotions gave better performance than the single modal method. In addition, hyperparameter tuning can affect the performance of multimodal systems.

show abstract

Section: Xlnet Transformersmentioning

confidence: 99%

Multimodal music emotion recognition in Indonesian songs based on CNN-LSTM, XLNet transformers

Sams

Zahra²

2023

Bulletin EEI

View full text Add to dashboard Cite

show abstract

“…Extensive research into emotion in songs has been carried out through Music and Multimedia Information Retrieval (MMIR), which focuses on “mood trajectories” in lyrics and other acoustic features (Hu et al, 2009; Nanayakkara & Caldera, 2016; Nishikawa et al, 2011; Yang & Lee, 2009). The most common analytical method for lyrics is “probabilistic latent semantic analysis” (PLSA), an intuitive approach based on “impression” and clustering of affective words into “types” of feelings (Nanayakkara & Caldera, 2016; Nishikawa et al, 2011, p. 51). This approach detects emotions explicitly expressed as emotive words.…”

Section: Review Of Related Researchmentioning

confidence: 99%

Exploring a systemic functional semiotics approach to understanding emotional expression in singing performance: Implications for music education

Ngo

Spreadborough

2021

Research Studies in Music Education

View full text Add to dashboard Cite

Engagement with songs through performance and analysis is a key component of music curricula worldwide. Music learning has a significant impact on a number of student competencies, including enhancing students’ communicative abilities as they learn to manipulate, express, and share sound in both voice qualities and lyrics. However, common analyses of singing performance rarely focus exclusively on voice quality, and there is no systematic framework which considers how emotional meaning in lyrics interacts with emotional meaning in voice quality. Drawing on systemic functional semiotics, this article proposes a unified theoretical framework for examining how emotional meaning is co-constructed in the voice and lyrics in singing performance. This framework provides a novel approach for discussing and teaching song analysis and performance. The framework will be illustrated through the analysis of the interaction between voice quality and lyrics in the song “Someone Like You” performed by Adele.

show abstract

Music Emotion Recognition With Audio and Lyrics Features

Cited by 2 publications

References 5 publications

Multimodal music emotion recognition in Indonesian songs based on CNN-LSTM, XLNet transformers

Multimodal music emotion recognition in Indonesian songs based on CNN-LSTM, XLNet transformers

Exploring a systemic functional semiotics approach to understanding emotional expression in singing performance: Implications for music education

Contact Info

Product

Resources

About