Polyphonic piano note transcription with recurrent neural networks

Abstract

In this paper a new approach for polyphonic piano note onset transcription is presented. It is based on a recurrent neural network to simultaneously detect the onsets and the pitches of the notes from spectral features. Long Short-Term Memory units are used in a bidirectional neural network to model the context of the notes. The use of a single regression output layer instead of the often used one-versus-all classification approach enables the system to significantly lower the number of erroneous note detections. Evaluation is based on common test sets and shows exceptional temporal precision combined with a significant boost in note transcription performance compared to current state-of-the-art approaches. The system is trained jointly with various synthesized piano instruments and real piano recordings and thus generalizes much better than existing systems.

Automatic music transcription: challenges and future directions

“…Other onset detection methods that have performed well in MIREX evaluations include the use of psychoacoustically motivated features [26], transient peak classification [114] and pitch-based features [129]. A data-driven approach using supervised learning, where various neural network architectures have been utilised, has given the best results in several MIREX evaluations, including the most recent one (2012) [17,47,79]. Finally, Degara et al [31] exploit rhythmic regularity in music using a probabilistic framework to improve onset detection, showing that the integration of onset detection with higher-level rhythmic processing is advantageous.…”

Note onset detection in musical signals via neural–network–based multi–ODF fusion

“…A moving average or a moving median is usually preferred over a fixed threshold as it can follow the dynamics of a sound (Duxbury et al, 2003;Böck et al, 2012). Additionally, some methods for controlling the salience of a peak are often applied (Dixon, 2006).…”

“…The opposite outcome (low recall and high precision) is expected for too high a threshold value, overshooting many relevant peaks. The harmonic mean of precision and recall, known as the F-measure, is therefore often reported as a "balanced" result of the onset detection procedure (Dixon, 2006;Böck et al, 2012).…”

“…The popularity of machine learning applications for onset detection is growing rapidly with some excellent results reported in recent research. Neural networks are the tool of choice (Lacoste and Eck, 2007;Böck et al, 2012), although other data-driven techniques have also been used (Davy and Godsill, 2002). The input data usually consist of a time-frequency representation of the sound signal, mapped non-linearly in the frequency domain according to a perceptual model.…”

“…The input data usually consist of a time-frequency representation of the sound signal, mapped non-linearly in the frequency domain according to a perceptual model. Böck et al (2012) used a bank of triangular filters positioned at critical bands of the Bark scale to filter the STFT magnitude spectra, computed with three different window lengths in parallel. In this way, the redundancy resulting from unnecessarily high frequency resolution of the STFT in the upper frequency range may be avoided.…”