Staff-line removal with selectional auto-encoders

Gallego, Antonio‐Javier; Calvo-Zaragoza, Jorge

doi:10.1016/j.eswa.2017.07.002

Cited by 29 publications

(16 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Montagner et al [26] followed a similar approach but learning the parameters of the morphological image operators. Calvo-Zaragoza et al [27] used Support Vector Machines to classify pixels belonging to the categories staff or non-staff, which was later extended by incorporating the use of CNN [28] and Auto-Encoders [29].…”

Section: Introductionmentioning

confidence: 99%

Deep Neural Networks for Document Processing of Music Score Images

et al. 2018

Self Cite

View full text Add to dashboard Cite

There is an increasing interest in the automatic digitization of medieval music documents. Despite efforts in this field, the detection of the different layers of information on these documents still poses difficulties. The use of Deep Neural Networks techniques has reported outstanding results in many areas related to computer vision. Consequently, in this paper, we study the so-called Convolutional Neural Networks (CNN) for performing the automatic document processing of music score images. This process is focused on layering the image into its constituent parts (namely, background, staff lines, music notes, and text) by training a classifier with examples of these parts. A comprehensive experimentation in terms of the configuration of the networks was carried out, which illustrates interesting results as regards to both the efficiency and effectiveness of these models. In addition, a cross-manuscript adaptation experiment was presented in which the networks are evaluated on a different manuscript from the one they were trained. The results suggest that the CNN is capable of adapting its knowledge, and so starting from a pre-trained CNN reduces (or eliminates) the need for new labeled data.

show abstract

Section: Introductionmentioning

confidence: 99%

Deep Neural Networks for Document Processing of Music Score Images

et al. 2018

Self Cite

View full text Add to dashboard Cite

show abstract

“…The sole difference in terms of data preparation is that for the scanned sheet music, we of course do not have the annotated system bounding boxes available. As the overall goal is to have the means to fully automatically index a large collection of scores, we developed an automatic system detection algorithm inspired by (Gallego and Calvo-Zaragoza, 2017;Dorfer et al, 2017b). Given the automatic system detection, we have all the tools to automatically create the database (cf.…”

Section: Methodsmentioning

confidence: 99%

Learning Audio–Sheet Music Correspondences for Cross-Modal Retrieval and Piece Identification

Dorfer¹,

Hajič²,

Arzt³

et al. 2018

Transactions of the International Society for Music Information Retrieval

View full text Add to dashboard Cite

This work addresses the problem of matching musical audio directly to sheet music, without any higherlevel abstract representation. We propose a method that learns joint embedding spaces for short excerpts of audio and their respective counterparts in sheet music images, using multimodal convolutional neural networks. Given the learned representations, we show how to utilize them for two sheet-music-related tasks: (1) piece/score identification from audio queries and (2) retrieving relevant performances given a score as a search query. All retrieval models are trained and evaluated on a new, large scale multimodal audio-sheet music dataset which is made publicly available along with this article. The dataset comprises 479 precisely annotated solo piano pieces by 53 composers, for a total of 1,129 pages of music and about 15 hours of aligned audio, which was synthesized from these scores. Going beyond this synthetic training data, we carry out first retrieval experiments using scans of real sheet music of high complexity (e.g., nearly the complete solo piano works by Frederic Chopin) and commercial recordings by famous concert pianists. Our results suggest that the proposed method, in combination with the large-scale dataset, yields retrieval models that successfully generalize to data way beyond the synthetic training data used for model building.

show abstract

“…That is why, for many years, much research was devoted to improving staff-line removal [21]. Currently, thanks to the use of deep neural networks, the staff-line removal can be considered a solved problem, with selectional auto-encoders outperforming all previously existing methods given a sufficient amount of training data [22]. However, even with an ideal staff-line removal algorithm, isolating musical symbols by means of connected components remains problematic, since multiple primitives could be connected to each other (e.g., a beam group can be a single connected component that includes several heads, stems, and beams) or a single unit can have multiple disconnected parts (e.g., a fermata, voltas, f-clef).…”

Section: Background On Music Object Detectionmentioning

confidence: 99%

A Baseline for General Music Object Detection with Deep Learning

2018

Self Cite

View full text Add to dashboard Cite

Deep learning is bringing breakthroughs to many computer vision subfields including Optical Music Recognition (OMR), which has seen a series of improvements to musical symbol detection achieved by using generic deep learning models. However, so far, each such proposal has been based on a specific dataset and different evaluation criteria, which made it difficult to quantify the new deep learning-based state-of-the-art and assess the relative merits of these detection models on music scores. In this paper, a baseline for general detection of musical symbols with deep learning is presented. We consider three datasets of heterogeneous typology but with the same annotation format, three neural models of different nature, and establish their performance in terms of a common evaluation standard. The experimental results confirm that the direct music object detection with deep learning is indeed promising, but at the same time illustrates some of the domain-specific shortcomings of the general detectors. A qualitative comparison then suggests avenues for OMR improvement, based both on properties of the detection model and how the datasets are defined. To the best of our knowledge, this is the first time that competing music object detection systems from the machine learning paradigm are directly compared to each other. We hope that this work will serve as a reference to measure the progress of future developments of OMR in music object detection.

show abstract

Staff-line removal with selectional auto-encoders

Cited by 29 publications

References 26 publications

Deep Neural Networks for Document Processing of Music Score Images

Deep Neural Networks for Document Processing of Music Score Images

Learning Audio–Sheet Music Correspondences for Cross-Modal Retrieval and Piece Identification

A Baseline for General Music Object Detection with Deep Learning

Contact Info

Product

Resources

About