Bytecover: Cover Song Identification Via Multi-Loss Training

Abstract

We present in this paper ByteCover, which is a new feature learning method for cover song identification (CSI). Byte-Cover is built based on the classical ResNet model, and two major improvements are designed to further enhance the capability of the model for CSI. In the first improvement, we introduce the integration of instance normalization (IN) and batch normalization (BN) to build IBN blocks, which are major components of our ResNet-IBN model. With the help of the IBN blocks, our CSI model can learn features that are invariant to the changes of musical attributes such as key, tempo, timbre and genre, while preserving the version information. In the second improvement, we employ the BN-Neck method to allow a multi-loss training and encourage our method to jointly optimize a classification loss and a triplet loss, and by this means, the inter-class discrimination and intra-class compactness of cover songs, can be ensured at the same time. A set of experiments demonstrated the effectiveness and efficiency of ByteCover on multiple datasets, and in the Da-TACOS dataset, ByteCover outperformed the best competitive system by 18.0%.

Analysis of Social Media Messaging for Health Promotion by the CDC from 2018 to2019 in Guangzhou, China (Preprint)

“…In the selfattention layer, we use a random mask to erase some value of the embedding to improve the robustness further. As a result, the introduced time-pooling module leverages feature statistics and pays more attention to the discriminative frames, which leads to clear outperformance compared to the widely-adopted generalized mean pooling (GeM) in recent approaches [6], [7], [10]. More ablation studies will be shown in section III-D.…”

“…We use the embedding before the bottleneck layer to compute the contrast loss (i.e., triplet loss [14]) and use the embedding after the bottleneck layer to compute the focal loss [15] and center loss [16]. Note that we use the focal loss to replace the traditional cross-entropy loss as in [6], improving the performance when facing data unbalance. Center loss [16] helps the training convergence and achieves higher performance.…”

“…Datasets. To fairly compare our method with others, we follow [6], [10] and use SHS100K [17] as the training dataset. We download the data with URLs from second-hand-songs website 1 and use the original setting 2 to split all recordings into train/dev/test sets.…”

“…For example, Yu et al proposed TPPNet (temporal pyramid pooling) [4] and CNN-based CQTNet [5] for learning the characteristics of the cover song task. Byte-Cover [6] and ByteCover2 [7] achieved state-of-the-art results with a ResNet-IBN50 [8] backbone and multi-loss training with cross-entropy and triplet loss. PiCKINet [9] proposed Pitch Class Blocks in order to maintain the key-invariance features of music.…”

“…To overcome such shortcomings, we propose to use the Conformer [11] to capture global information with the selfattention mechanism while still accounting for local clues with convolutions. Moreover, when pooling the features across the time domain to obtain a fixed-length audio embedding, we abandon the popular choice of average pooling or generalized mean pooling [6] that treats the features equivalently across time. Instead, we introduce a self-attention mechanism to amplify the contribution of discriminative frames and suppress redundant information that is unrelated to identifying cover songs.…”