Hybrid hidden Markov models and artificial neural networks for handwritten music recognition in mensural notation

Abstract

In this paper we present a hybrid approach using Hidden Markov Models (HMM) and Artificial Neural Networks to deal with the task of Handwritten Music Recognition in Mensural notation. Previous works have shown that the task can be addressed with Gaussian density HMMs that can be trained and used in an end-to-end manner; that is, without prior segmentation of the symbols. However, the results achieved using that approach are not sufficiently accurate to be useful in practice. In this work we hybridize HMMs with deep Multi-Layer Perceptrons (MLP), which lead to remarkable improvements in optical symbol modeling. Moreover, this hybrid architecture maintains important advantages of HMMs such as the ability of properly modeling variable-length symbol sequences through segmentation-free training, and the simplicity and robustness of combining optical models with N-gram language models, which provide statistical a priori information about regularities in musical symbol concatenation observed in the training data. The results obtained with the proposed hybrid MLP-HMM approach outperform previous works by a wide margin, achieving symbol-level error rates around 26%, as compared with about 40% reported in previous works.

Handwritten Music Recognition for Mensural notation with convolutional recurrent neural networks

“…In addition, the accuracy of the considered approach is compared with previous work over the same corpus. Specifically: HMMs both trained with the classic Maximum Likelihood (ML) estimation (Calvo-Zaragoza et al, 2016) and with Discriminative Training (DT) (Calvo-Zaragoza et al, 2017), as well as HMMs hybridized with Multi-Layer Perceptron (MLP) models (Calvo-Zaragoza et al, 2019). In these cases, we directly consider the results with the best N-gram estimation found in the aforementioned references.…”

“…Concerning this formulation, Pugin (2006) already proposed a holistic approach for printed Mensural notation using Hidden Markov Models (HMM). This approach was recently extended to handwritten sources by using a more appropriate set of features (Calvo-Zaragoza et al, 2016), and further improved by considering discriminative training techniques (Calvo-Zaragoza et al, 2017), as well as hybridization with neural networks (Calvo-Zaragoza et al, 2019).…”

Lithofacies classification based on a hybrid system of artificial neural networks and hidden Markov models

“…fixed; Ulrych et al 2001); Pr(S) is straightforwardly estimated from the relative frequencies of lithofacies in the training data set; and Pr(S|Y) is the output of ANN, as calculated by the softmax equation. Note that any explicitly or implicitly computed Pr(S|Y) from statistically interpretable classifier can be easily adapted to model Pr(Y|S) (Calvo-Zaragoza et al 2019).…”

Improved barnacles mating optimizer algorithm for feature selection and support vector machine optimization

“…For supervised learning, the training samples include inputs and outputs (i.e., features and class labels), which results in a better result than unsupervised learning in most cases [ 12 ]. The supervised algorithm commonly used includes decision tree (DT) [ 13 ], naïve Bayes (NB) [ 14 ], k-nearest neighbor (kNN) [ 15 – 17 ], neural networks (NNs) [ 18 , 19 ], and support vector machine (SVM) [ 20 – 22 ]. Among them, SVM was first formally proposed by Cortes and Vapnik in 1995.…”