Region Normalized Capsule Network Based Generative Adversarial Network for Non-parallel Voice Conversion

Akhter, Md. Tousin; Banerjee, Padmanabha; Dhar, Sandipan; Ghosh, Subhayu; Jana, Nanda Dulal

doi:10.1007/978-3-031-48309-7_20

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2024

Publication Types

Select...

Article2

Book1

Relationship

Self Cite0

Independent3

Authors

Journals

Cited by 3 publications

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

A Comprehensive Analysis on Features and Performance Evaluation Metrics in Audio-Visual Voice Conversion

Ghosh,

Dhar,

Jana

2024

Communications in Computer and Information Science

View full text Add to dashboard Cite

A Comprehensive Analysis on Features and Performance Evaluation Metrics in Audio-Visual Voice Conversion

Ghosh,

Dhar,

Jana

2024

Communications in Computer and Information Science

View full text Add to dashboard Cite

Audio-visual speech synthesis using vision transformer–enhanced autoencoders with ensemble of loss functions

Ghosh,

Sarkar,

Ghosh

et al. 2024

Appl Intell

View full text Add to dashboard Cite

CCLCap-AE-AVSS: Cycle consistency loss based capsule autoencoders for audio–visual speech synthesis

Ghosh,

Jana,

et al. 2024

Journal of Intelligent Systems

View full text Add to dashboard Cite

Audio–visual speech synthesis (AVSS) is a rapidly growing field in the paradigm of audio–visual learning, involving the conversion of one person’s speech into the audio–visual stream of another while preserving the speech content. AVSS comprises two primary components: voice conversion (VC), which alters the vocal characteristics from the source speaker to the target speaker, followed by audio–visual synthesis, which creates the audio–visual presentation of the converted VC output for the target speaker. Despite the progress in deep learning (DL) technologies, DL models in AVSS have received limited attention in existing literature. Therefore, this article presents a novel approach for AVSS utilizing capsule network (Caps-Net)-based autoencoders, with the incorporation of cycle consistency loss. Caps-Net addresses translation invariance issues in convolutional neural network approaches for effective feature capture. Additionally, the inclusion of cycle consistency loss ensures the retention of content information from the source speaker. The proposed approach is referred to as cycle consistency loss-based capsule autoencoders for audio–visual speech synthesis (CCLCap-AE-AVSS). The proposed CCLCap-AE-AVSS is trained and tested using VoxCeleb2 and LRS3-TED datasets. The subjective and objective assessments of the generated samples demonstrate the superior performance of the proposed work compared to the current state-of-the-art models.

show abstract

Region Normalized Capsule Network Based Generative Adversarial Network for Non-parallel Voice Conversion

Cited by 3 publications

References 19 publications

A Comprehensive Analysis on Features and Performance Evaluation Metrics in Audio-Visual Voice Conversion

A Comprehensive Analysis on Features and Performance Evaluation Metrics in Audio-Visual Voice Conversion

Audio-visual speech synthesis using vision transformer–enhanced autoencoders with ensemble of loss functions

CCLCap-AE-AVSS: Cycle consistency loss based capsule autoencoders for audio–visual speech synthesis

Contact Info

Product

Resources

About