Deep Neural Networks for Document Processing of Music Score Images

Calvo-Zaragoza, Jorge; Castellanos, Francisco J.; Vigliensoni, Gabriel; Fujinaga, Ichiro

doi:10.3390/app8050654

Cited by 32 publications

(23 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among others, the project deals with the music scores in the Salzinnes Antiphonal manuscript dated to the middle of the 16th century; this is the data that will be dealt with within this work. Different schemes have been used to process and analyze these data: convolutional neural networks have been employed for binarization [5], categorization of score elements [14,21] and layering into their constituent elements [22]. An example of the application of this analysis process is shown in Figure 1, where the original image (a), staves (b), background (c) and symbols extracted (d) are shown for a sample score.…”

Section: Context: Simssa Projectmentioning

confidence: 99%

Automatic Staff Reconstruction within SIMSSA Project

Tardón

Barbancho

et al. 2020

Applied Sciences

Self Cite

View full text Add to dashboard Cite

The automatic analysis of scores has been a research topic of interest for the last few decades and still is since music databases that include musical scores are currently being created to make musical content available to the public, including scores of ancient music. For the correct analysis of music elements and their interpretation, the identification of staff lines is of key importance. In this paper, a scheme to post-process the output of a previous musical object identification system is described. This system allows the reconstruction by means of detection, tracking and interpolation of the staff lines of ancient scores from the digital Salzinnes Database. The scheme developed shows a remarkable performance on the specific task it was created for.

show abstract

Section: Context: Simssa Projectmentioning

confidence: 99%

Automatic Staff Reconstruction within SIMSSA Project

Tardón

Barbancho

et al. 2020

Applied Sciences

Self Cite

View full text Add to dashboard Cite

show abstract

“…For instance the first NC of the second last neume in Figure 1 might also be an G instead of an F. In this work, we do not distinguish additional note types such as liquecents or oriscus. Furthermore, unlike the approach in [25], we do not label the images on a pixel-basis, since detecting the actual shape and extent of neumes, clefs, or accidentals is out of the focus of this paper. We are solely interested in transcribing historical neume notations in a digitised form that preserves all melodic information which is the desired output in almost all cases.…”

Section: Datasetmentioning

confidence: 99%

Staff, Symbol, and Melody Detection of Medieval Manuscripts Written in Square Notation Using Deep Fully Convolutional Networks

Wick¹,

Hartelt²,

Puppe³

2019

Preprint

View full text Add to dashboard Cite

Even today, the automatic digitisation of scanned documents in general but especially the automatic optical music recognition (OMR) of historical manuscripts still remain an enormous challenge, since both handwritten musical symbols and text have to be identified. This paper focuses on the Medieval so-called square notation developed in the 11th-12th century, which is already composed of staff lines, staves, clefs, accidentals, and neumes, that are roughly spoken connected single notes. The aim is to develop an algorithm that captures both the neume and pitch, that is melody information that can be used to reconstruct the original writing. Our pipeline is similar to the standard OMR approach and comprises a novel staff line and symbol detection algorithm, based on deep Fully Convolutional Networks (FCN), which perform pixel-based predictions for either staff lines or symbols and their respective types. Then, the staff line detection combines the extracted lines to staves and yields an F1-score of over 99% for both detecting lines and complete staves. For the music symbol detection we choose a novel approach that skips the step to identify neumes and instead directly predicts note components (NCs) and their respective affiliation to a neume. Furthermore, the algorithm detects clefs and accidentals. Our algorithm recognises these symbols with an F1-score of over 96% if the type is ignored and predicts the true symbol sequence of a staff with a diplomatic symbol accuracy rate (dSAR) of about 87%. If only the NCs without their respective connection to a neume, all clefs, and accidentals are of interest the algorithm reaches an harmonic symbol accuracy rate (hSAR) of approximately 90%.

show abstract

“…Although the interest in OMR has reawakened with the appearance of deep learning, as far as we know, the few existing methods that attempt to recognize handwritten scores are mostly focused on solving a particular stage of OMR, such as layout analysis [3] or detection and classification of graphic primitives [4] or music symbols [5,6]. However, in the particular case of Western classical music, music scores are complex documents composed of staves (five horizontal lines), music symbols (e.g.…”

Section: Introductionmentioning

confidence: 99%

From Optical Music Recognition to Handwritten Music Recognition: A baseline

Baró

Riba

Calvo-Zaragoza

et al. 2019

Pattern Recognition Letters

Self Cite

View full text Add to dashboard Cite

Optical Music Recognition (OMR) is the branch of document image analysis that aims to convert images of musical scores into a computer-readable format. Despite decades of research, the recognition of handwritten music scores, concretely the Western notation, is still an open problem, and the few existing works only focus on a specific stage of OMR. In this work, we propose a full Handwritten Music Recognition (HMR) system based on Convolutional Recurrent Neural Networks, data augmentation and transfer learning, that can serve as a baseline for the research community.

show abstract

Deep Neural Networks for Document Processing of Music Score Images

Cited by 32 publications

References 34 publications

Automatic Staff Reconstruction within SIMSSA Project

Automatic Staff Reconstruction within SIMSSA Project

Staff, Symbol, and Melody Detection of Medieval Manuscripts Written in Square Notation Using Deep Fully Convolutional Networks

From Optical Music Recognition to Handwritten Music Recognition: A baseline

Contact Info

Product

Resources

About