Efficient Query-by-Content Audio Retrieval by Locality Sensitive Hashing and Partial Sequence Comparison

Abstract

SUMMARYThis paper investigates suitable indexing techniques to enable efficient content-based audio retrieval in large acoustic databases. To make an index-based retrieval mechanism applicable to audio content, we investigate the design of Locality Sensitive Hashing (LSH) and the partial sequence comparison. We propose a fast and efficient audio retrieval framework of query-by-content and develop an audio retrieval system. Based on this framework, four different audio retrieval schemes, LSH-Dynamic Programming (DP), LSH-Sparse DP (SDP), Exact Euclidian LSH (E2LSH)-DP, E2LSH-SDP, are introduced and evaluated in order to better understand the performance of audio retrieval algorithms. The experimental results indicate that compared with the traditional DP and the other three compititive schemes, E2LSH-SDP exhibits the best tradeoff in terms of the response time, retrieval accuracy and computation cost. key words: indexing, locality-sensitive hashing, content-based audio retrieval, dynamic programming 1. Introduction Content-based audio retrieval is not only a very promising research topic but also one of the main problems, in multimedia information processing. Handling audio sequence data is usually time-consuming due to the high dimensionality of the features, which makes it inconvenient to utilize the potential content-based information retrieval techniques on the Internet or personal media devices. To access a huge mass of audio information efficiently, it is necessary to explore the audio information, facilitate the management of audio data and serve multimedia applications. Consequently various indexing structures have been reported in the study of audio retrieval. These include, for example, hierarchical structure As far as the creation of query-by-content audio retrieval mechanism via indexing techniques is concerned the main challenges are as follows: (1)

Local summarization and multi-level LSH for retrieving multi-variant audio tracks

“…This can be performed by directly looking for audio tracks whose content descriptions are similar to the description of the query [2,3,4], according to some relevant similarity measures. It is also possible to first map the query music to some related category (e.g.…”

“…The detection of multi-variant musical audio tracks is considered in previous work (see [1,2,3,4]) as a sub-topic of query-by-content in MIR. It is an interesting and motivating subject, especially with more and more unknown audio recordings being uploaded to User Generated Content (UGC) websites.…”

“…Regarding the first issue, the aim is to improve the accuracy of multi-variant audio track detection and the different proposals rely on pitch [9,10], Mel-Frequency Cepstral Coefficients (MFCC) [11,12,13] or Chroma [2,14]. With regard to the latter research issue, the goal is to accelerate the retrieval by similarity and the existing proposals include tree structures [7,15,16], other hierarchical structures [17], LSH [4,11,18], Exact Euclidean LSH (E 2 LSH) [3,4] and other variants of LSH [6,11]. It is however clear that the two research issues are not independent, since more accurate detection requires more elaborate representations of audio content, with a negative impact on scalability.…”

“…The effective and efficient detection of multi-variant audio tracks is thus a motivating topic in Music Information Retrieval (MIR), related to musical audio signal processing [1,2], audio content indexing [3,4], sequence matching [1,4,5], music retrieval evaluation [6], etc. It has already attracted significant research interest, see e.g.…”

“…It has already attracted significant research interest, see e.g. [1,2,3,4]. The previous proposals rely on comparing either sequences of audio frame descriptions (feature sequences) [1,2] or on extracting very compact descriptions of entire audio sequences [6,7].…”