Decoupling music notation to improve end-to-end Optical Music Recognition

Alfaro-Contreras, María; Ríos-Vila, Antonio; Valero-Mas, Jose J.; Iñesta, José M.; Calvo-Zaragoza, Jorge

doi:10.1016/j.patrec.2022.04.032

Cited by 7 publications

(6 citation statements)

References 23 publications

(38 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Handwritten music recognition focuses on extracting and interpreting musical notations from handwritten manuscripts, including complexities such as varying handwriting styles and potential ambiguities. In recent years, there have been some breakthroughs in the research of handwritten music recognition with deep learning methods [8][9][10]14,15]. In contrast, printed music score recognition involves analyzing and extracting musical symbols and annotations from printed music scores, typically characterized by standardized fonts and precise formatting.…”

Section: Related Workmentioning

confidence: 99%

A Stave-Aware Optical Music Recognition on Monophonic Scores for Camera-Based Scenarios

Liu,

Wu,

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

The recognition of printed music sheets in camera-based realistic scenarios is a novel research branch of optical music recognition (OMR). However, special factors in realistic scenarios, such as uneven lighting distribution and curvature of staff lines, can have adverse effects on OMR models designed for digital music scores. This paper proposes a stave-aware method based on object detection to recognize monophonic printed sheet music in camera-based scenarios. By detecting the positions of staff lines, we improve the accuracy of note pitch effectively. In addition, we present the Camera Printed Music Staves (CPMS) dataset, which consists of labels and images captured by mobile phones under different angles and lighting conditions in realistic scenarios. We compare our method after training on different datasets with a sequence recognition method called CRNN-CTC on the test set of the CPMS dataset. The results show that the accuracy, robustness, and data dependency of our method perform better.

show abstract

Section: Related Workmentioning

confidence: 99%

A Stave-Aware Optical Music Recognition on Monophonic Scores for Camera-Based Scenarios

Liu,

Wu,

et al. 2023

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…The disadvantage of RNN is that it cannot solve the problem of long-term dependence, and there is a phenomenon of network gradient dissipation and explosion. Against the defect of RNN, the LSTM NN is proposed (Liu et al, 2021 ; Alfaro-Contreras et al, 2022 ), as drawn in Figure 3 .…”

Section: Design Of Music Style Recognition Model and Construction Of ...mentioning

confidence: 99%

Recognition of musical beat and style and applications in interactive humanoid robot

Chu

2022

Front. Neurorobot.

View full text Add to dashboard Cite

The musical beat and style recognition have high application value in music information retrieval. However, the traditional methods mostly use a convolutional neural network (CNN) as the backbone and have poor performance. Accordingly, the present work chooses a recurrent neural network (RNN) in deep learning (DL) to identify musical beats and styles. The proposed model is applied to an interactive humanoid robot. First, DL-based musical beat and style recognition technologies are studied. On this basis, a note beat recognition method combining attention mechanism (AM) and independent RNN (IndRNN) [AM-IndRNN] is proposed. The AM-IndRNN can effectively avoid gradient vanishing and gradient exploding. Second, the audio music files are divided into multiple styles using the music signal's temporal features. A human dancing robot using a multimodal drive is constructed. Finally, the proposed method is tested. The results show that the proposed AM-IndRNN outperforms multiple parallel long short-term memory (LSTM) models and IndRNN in recognition accuracy (88.9%) and loss rate (0.0748). Therefore, the AM-optimized LSTM model has gained a higher recognition accuracy. The research results provide specific ideas for applying DL technology in musical beat and style recognition.

show abstract

“…Besides, the Connectionist Temporal Classification (CTC) is proposed firstly for training RNNs to label unsegmented sequence directly [15]. Then it was used to train deep learning models for OMR with an end-to-end manner in [16,17].…”

Section: Introductionmentioning

confidence: 99%

A Sequence-to-Sequence Framework Based on Transformer With Masked Language Model for Optical Music Recognition

Wen

Zhu

2022

IEEE Access

View full text Add to dashboard Cite

Optical music recognition technology is of great significance in the development of the digital music. In recent years, the convolutional recurrent neural network framework with connectionist temporal classification has been used in music recognition. However, its loss function is calculated in serial mode, which leads to low efficiency in training and difficulty in convergence. Additionally, because of the gradient disappearance of excessive long music sequences, the existing music recognition models are hard to learn the relationships between musical symbols, resulting in high sequence error rate. Therefore, we propose a sequence-to-sequence framework based on transformer with masked language model to deal with these problems. The context representation between musical symbols can be captured further by the self-attention module in the transformer, which will reduce the sequence error rate. In addition, we refer to the masked language model and design a mask matrix to predict each musical symbol in a parallel way, so as to speed up the training process. Our experiments are carried out on the printed images of music stave dataset, and the results show that our proposed method is training-efficient and has great improvement in sequence accuracy rate.

show abstract

Decoupling music notation to improve end-to-end Optical Music Recognition

Cited by 7 publications

References 23 publications

A Stave-Aware Optical Music Recognition on Monophonic Scores for Camera-Based Scenarios

A Stave-Aware Optical Music Recognition on Monophonic Scores for Camera-Based Scenarios

Recognition of musical beat and style and applications in interactive humanoid robot

A Sequence-to-Sequence Framework Based on Transformer With Masked Language Model for Optical Music Recognition

Contact Info

Product

Resources

About