A Mid-Level Representation for Melody-Based Retrieval in Audio Collections

Abstract

Searching audio collections using high-level musical descriptors is a difficult problem, due to the lack of reliable methods for extracting melody, harmony, rhythm, and other such descriptors from unstructured audio signals. In the paper, we present a novel approach to melody-based retrieval in audio collections. Our approach supports audio, as well as symbolic queries and ranks results according to melodic similarity to the query. We introduce a beat-synchronous melodic representation consisting of salient melodic lines, which are extracted from the analyzed audio signal. We propose the use of a two-dimensional shift-invariant transform to extract shift-invariant melodic fragments from the melodic representation and demonstrate how such fragments can be indexed and stored in a song database. An efficient search algorithm based on locality-sensitive hashing is used to perform retrieval according to similarity of melodic fragments. On the cover song detection task, good results are achieved for audio, as well as for symbolic queries, while fast retrieval performance makes the proposed system suitable for retrieval in large databases.

Audio Cover Song Identification and Similarity: Background, Approaches, Evaluation, and Beyond

“…Melody is a salient musical descriptor of a piece of music [73] and, therefore, several cover song identification systems use melody representations as a main descriptor [49,50,68,78,79]. As a first processing step, these systems need to extract the This is a preliminary draft.…”

“…To refine the obtained representation, cover detection systems usually need to combine a melody extractor with a voice/non-voice detector and other post-processing modules in order to achieve a more reliable representation [68,78,79]. Another possibility is to generate a so-called "mid-level" representation for these melodies [49,50], where the emphasis is not only put on melody extraction, but also on the feasibility to describe audio in a way that facilitates retrieval.…”

“…The most straightforward way to achieve key invariance is to test all possible feature transpositions [21,23,36,38,39,41,50,55]. In the case of an octaveindependent representation, this implies the computation of a similarity measure for all possible circular (or ring) shifts in the pitch axis for each song.…”

“…A very interesting approach to achieve transposition invariance is to use a 2D power spectrum [50] or a 2D autocorrelation function [37]. Autocorrelation is a well-known operator for converting signals into a delay or shift-invariant representation [58].…”

“…This can be done independently of the descriptor used. Some cover song identification systems based on a PCP [23,55] or a melodic [49,50] representation use this strategy, and extensions with chords or other types of information could be easily devised. If the beat does not provide enough temporal resolution, a finer representation (e.g.…”