EmoMV: Affective music-video correspondence learning datasets for classification and retrieval

Thao, Ha Thi Phuong; Roig, Gemma; Herremans, Dorien

doi:10.1016/j.inffus.2022.10.002

Cited by 7 publications

(3 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Alternatively, in an effort to collect larger quantities of affect labels in a shorter amount of time, although with a potential loss in accuracy, crowd-sourcing on platforms such as Amazon Mechanical Turk (MTurk) has also been explored [ 3 , 53 , 54 , 55 , 56 ]. Some researchers utilize a mix of both online and offline collection methods [ 57 , 58 ], or even use predictive models such as AttendAffectNet [ 59 ] for the emotion labeling [ 60 ]. Regardless of the data collection method, it is important for each musical excerpt in the dataset to be labelled by multiple participants in order to account for subjectivity.…”

Section: Data Gathering Proceduresmentioning

confidence: 99%

“…The models are simple in design and intended to be supplementary performance benchmarks on the dataset. In future work, more state-of-the-art methods such as convolutional neural networks [ 70 , 71 , 72 , 73 , 74 , 75 , 76 ], or transformer architectures [ 60 , 77 , 78 ] could be used with the dataset for further experimentation with profile information and its uses for building improved MER models.…”

Section: Emotion Prediction Modelsmentioning

confidence: 99%

See 1 more Smart Citation

MERP: A Music Dataset with Emotion Ratings and Raters’ Profile Information

Koh

Cheuk

Heung

et al. 2022

Sensors

Self Cite

View full text Add to dashboard Cite

Music is capable of conveying many emotions. The level and type of emotion of the music perceived by a listener, however, is highly subjective. In this study, we present the Music Emotion Recognition with Profile information dataset (MERP). This database was collected through Amazon Mechanical Turk (MTurk) and features dynamical valence and arousal ratings of 54 selected full-length songs. The dataset contains music features, as well as user profile information of the annotators. The songs were selected from the Free Music Archive using an innovative method (a Triple Neural Network with the OpenSmile toolkit) to identify 50 songs with the most distinctive emotions. Specifically, the songs were chosen to fully cover the four quadrants of the valence-arousal space. Four additional songs were selected from the DEAM dataset to act as a benchmark in this study and filter out low quality ratings. A total of 452 participants participated in annotating the dataset, with 277 participants remaining after thoroughly cleaning the dataset. Their demographic information, listening preferences, and musical background were recorded. We offer an extensive analysis of the resulting dataset, together with a baseline emotion prediction model based on a fully connected model and an LSTM model, for our newly proposed MERP dataset.

show abstract

Section: Data Gathering Proceduresmentioning

confidence: 99%

Section: Emotion Prediction Modelsmentioning

confidence: 99%

MERP: A Music Dataset with Emotion Ratings and Raters’ Profile Information

Koh

Cheuk

Heung

et al. 2022

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…Developments in multimedia technology have resulted in a sharp increase in the variety of digital music and its listening volume, necessitating urgent advancements in music information retrieval (MIR), which involves utilizing computer technology to automatically analyze, recognize, retrieve, and understand music. Audio music genre classification is a MIR task that involves assigning labels to each piece of music based on characteristics such as genre [1,2], mood [3,4], and artist type [5,6]. Audio music genre classification enables the automatic categorization of audio music based on different styles or types, facilitating a deeper understanding and organization of music libraries.…”

Section: Introductionmentioning

confidence: 99%

Deformer: Denoising Transformer for Improved Audio Music Genre Classification

Wang,

Li,

Sung

2023

Applied Sciences

View full text Add to dashboard Cite

Audio music genre classification is performed to categorize audio music into various genres. Traditional approaches based on convolutional recurrent neural networks do not consider long temporal information, and their sequential structures result in longer training times and convergence difficulties. To overcome these problems, a traditional transformer-based approach was introduced. However, this approach employs pre-training based on momentum contrast (MoCo), a technique that increases computational costs owing to its reliance on extracting many negative samples and its use of highly sensitive hyperparameters. Consequently, this complicates the training process and increases the risk of learning imbalances between positive and negative sample sets. In this paper, a method for audio music genre classification called Deformer is proposed. The Deformer learns deep representations of audio music data through a denoising process, eliminating the need for MoCo and additional hyperparameters, thus reducing computational costs. In the denoising process, it employs a prior decoder to reconstruct the audio patches, thereby enhancing the interpretability of the representations. By calculating the mean squared error loss between the reconstructed and real patches, Deformer can learn a more refined representation of the audio data. The performance of the proposed method was experimentally compared with that of two distinct baseline models: one based on S3T and one employing a residual neural network-bidirectional gated recurrent unit (ResNet-BiGRU). The Deformer achieved an 84.5% accuracy, surpassing both the ResNet-BiGRU-based (81%) and S3T-based (81.1%) models, highlighting its superior performance in audio classification.

show abstract