Avqvc: One-Shot Voice Conversion By Vector Quantization With Applying Contrastive Learning

Huaizhen, Tang,; Zhang, Xulong; Wang, Jianzong; Cheng, Ning; Xiao, Jing

doi:10.1109/icassp43922.2022.9746369

Cited by 36 publications

(7 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It makes the model more robust to interference such as noise and reverberation. Not only the speaker identification but also some other voice domain tasks, text-to-speech, voice conversion, etc., embedded X-vector for performance gains [33]- [36].…”

Section: B Timbral Featuresmentioning

confidence: 99%

Singer Identification for Metaverse with Timbral and Middle-Level Perceptual Features

Zhang¹,

Wang²,

Cheng³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Metaverse is an interactive world that combines reality and virtuality, where participants can be virtual avatars. Anyone can hold a concert in a virtual concert hall, and users can quickly identify the real singer behind the virtual idol through the singer identification. Most singer identification methods are processed using the frame-level features. However, expect the singer's timbre, the music frame includes music information, such as melodiousness, rhythm, and tonal. It means the music information is noise for using frame-level features to identify the singers. In this paper, instead of only the frame-level features, we propose to use another two features that address this problem. Middle-level feature, which represents the music's melodiousness, rhythmic stability, and tonal stability, and is able to capture the perceptual features of music. The timbre feature, which is used in speaker identification, represents the singers' voice features. Furthermore, we propose a convolutional recurrent neural network (CRNN) to combine three features for singer identification. The model firstly fuses the frame-level feature and timbre feature and then combines middle-level features to the mix features. In experiments, the proposed method achieves comparable performance on an average F1 score of 0.81 on the benchmark dataset of Artist20, which significantly improves related works.

show abstract

Section: B Timbral Featuresmentioning

confidence: 99%

Singer Identification for Metaverse with Timbral and Middle-Level Perceptual Features

Zhang¹,

Wang²,

Cheng³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Fixed-length speaker representation is obtained from pre-trained SV models [9]- [12], [15], or extracted from the target speaker's speech through disentanglement techniques such as instance normalization [13], [14], adversarial training [15], information bottleneck [9], [19], [20], or perturbation [10], [18]. To integrate the speaker representation with other speech components for speech generation, addition [16], concatenation [9], [19], or adaptive instance normalization [13], [14] are commonly used.…”

Section: A Typical Zero-shot Vc Frameworkmentioning

confidence: 99%

JDE multi-object tracking algorithm integrating multi-level semantic enhancement

Wang

Wang²,

Li³

et al. 2022

西北工业大学学报

View full text Add to dashboard Cite

In order to solve the problem of target ID switching caused by target occlusion and insufficient ID information and location information extraction in JDE(joint detection and embedding) algorithm, an improved multi-target tracking algorithm based on JDE is proposed in this paper. Firstly, the SPA feature space pyramid attention module is used to expand the receptive field and obtain more abundant semantic information to improve the detection accuracy of the model for different scale targets. Secondly, the FCN network makes the header and ID Embedding task collaborative learning to alleviate the excessive competition and enhance the original semantic information, effectively reducing the number of ID switching. Finally, PCCs-Ma motion measurement can strengthen the connection between Kalman filtering prediction and observation, and improve the reliability of similarity discrimination of motion characteristics. In order to verify the effectiveness of the algorithm, the JDE algorithm and the proposed algorithm are compared in the same experimental environment. The experimental results show that the average accuracy of model detection is improved by 3.94 %. On the MOT16 dataset, the MOTA and IDF1 indexes are increased by 6.9 %, and the number of ID switching of the improved algorithm is significantly reduced, and good tracking results are achieved.

show abstract

“…Thus, the identically distributed assumption is violated, halting the forthright application of the model trained in the su-pervised manner on the target unlabeled dataset. In view of that, the contrastive learning paradigm has achieved impressive successes in a spectrum of practical fields, such as computer vision (Wu et al 2018;Chen et al 2020;Qiang et al 2022), natural language processing (Gao, Yao, and Chen 2021;Qu et al 2021), signal processing (Yao et al 2022;Tang et al 2022), etc. As tractable surrogates, the contrastive approaches further establish promising capacity in the field of unsupervised graph learning, namely GCL methods (You et al 2020).…”

Section: Introductionmentioning

confidence: 99%

“…Another important issue with GCL is negative sampling. BGRL (Thakoor et al 2022) proposes a method that uses simple graph augmentation and does not have to construct negative samples while effectively scaling to large-scale graphs. To address the issue of sampling bias in GCL, PGCL (Lin et al 2021) proposes a negative sampling method based on semantic clustering.…”

Section: Introductionmentioning

confidence: 99%

Hierarchical Topology Isomorphism Expertise Embedded Graph Contrastive Learning

Li,

Jin,

Gao

et al. 2024

AAAI

View full text Add to dashboard Cite

Graph contrastive learning (GCL) aims to align the positive features while differentiating the negative features in the latent space by minimizing a pair-wise contrastive loss. As the embodiment of an outstanding discriminative unsupervised graph representation learning approach, GCL achieves impressive successes in various graph benchmarks. However, such an approach falls short of recognizing the topology isomorphism of graphs, resulting in that graphs with relatively homogeneous node features cannot be sufficiently discriminated. By revisiting classic graph topology recognition works, we disclose that the corresponding expertise intuitively complements GCL methods. To this end, we propose a novel hierarchical topology isomorphism expertise embedded graph contrastive learning, which introduces knowledge distillations to empower GCL models to learn the hierarchical topology isomorphism expertise, including the graph-tier and subgraph-tier. On top of this, the proposed method holds the feature of plug-and-play, and we empirically demonstrate that the proposed method is universal to multiple state-of-the-art GCL models. The solid theoretical analyses are further provided to prove that compared with conventional GCL methods, our method acquires the tighter upper bound of Bayes classification error. We conduct extensive experiments on real-world benchmarks to exhibit the performance superiority of our method over candidate GCL methods, e.g., for the real-world graph representation learning experiments, the proposed method beats the state-of-the-art method by 0.23% on unsupervised representation learning setting, 0.43% on transfer learning setting. Our code is available at https://github.com/jyf123/HTML.

show abstract

Avqvc: One-Shot Voice Conversion By Vector Quantization With Applying Contrastive Learning

Cited by 36 publications

References 16 publications

Singer Identification for Metaverse with Timbral and Middle-Level Perceptual Features

Singer Identification for Metaverse with Timbral and Middle-Level Perceptual Features

JDE multi-object tracking algorithm integrating multi-level semantic enhancement

Hierarchical Topology Isomorphism Expertise Embedded Graph Contrastive Learning

Contact Info

Product

Resources

About