Two systems for automatic music genre recognition

Abstract

We re-implement two state-of-the-art systems for music genre recognition, and closely examine their behavior. First, we find specific excerpts each system consistently and persistently mislabels. Second, we test the robustness of each system to spectral adjustments to audio signals. Finally, we expose the internal genre models of each system by testing if human can recognize the genres of music excerpts composed by each system to be highly genre-representative. Our results suggest that, though they have high mean classification accuracies, neither system is recognizing music genre.

The State of the Art Ten Years After a State of the Art: Future Research in Music Information Retrieval

“…This article rigorously addresses all of these, significantly extending our analysis of GTZAN in several practical ways. Our work not only illuminates results reported by a significant amount of work, but also provides a critical piece to address the non-trivial problems associated with evaluating music machine listening systems in valid ways [1,83,84,[103][104][105].…”

“…This article is not concerned with the validity, well-posedness, value, usefulness, or applicability of MGR; or whether MGR is "replaced by," or used in the service of, e.g., music similarity, autotagging, or the like. It is not concerned with making general conclusions about or criticisms of MGR or evaluation in MGR, which are comprehensively addressed in other works, e.g., [1,83,84,[105][106][107][108][109]. Finally, it is not concerned with how, or even whether it is possible, to create faultless datasets for MGR, music similarity, autotagging, and the like.…”

“…One might argue, "this article only touches the possibility of a critique of genre classification"; or, "it misses the valuable chance to criticize the simplistic and superficial approach to music that governs most of these publications"; or, "it falls short of providing a more rigorous questioning of MGR evaluation and a more general evaluation that goes beyond the GTZAN dataset." Such aspects, however, are outside the scope of this article delimited above, but are thoroughly addressed by other work, e.g., [1,83,84,105,[107][108][109].…”

“…Of these, one work explicitly endorses its integrity for MGR evaluation, while four allude to some problems. We find no work (outside our own [83][84][85][86]) that explicitly considers the musical content of GTZAN in evaluation.…”

“…We find that of these 100 works, more appear in 2010 -2012 than in the eight years after its creation. We find only five works (outside our own [83][84][85][86], and [99] which references [103]) that indicate someone has listened to some of GTZAN. Of these, one work explicitly endorses its integrity for MGR evaluation, while four allude to some problems.…”