Many-to-Many Voice Conversion Using Cycle-Consistent Variational Autoencoder with Multiple Decoders

Yook, Dongsuk; Leem, Seong-Gyun; Lee, Keonnyeong; Yoo, In-Chul

doi:10.21437/odyssey.2020-31

Cited by 11 publications

(8 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Compared with the presented models, the approach developed in this paper is a deterministic uneven autoencoder using a single encoder to create a latent representation of the ADS-B data linked to several decoders, each getting different data chosen thanks to a discriminating feature. This idea was used by Yook et al (2020) to separate the sound received by speakers placed differently but to the best of our knowledge, was never used in the anomaly detection field. As a result, the latent space created from the single encoder well represents the ADS-B data while the different specialized decoders well capture the information, addressing the variability of the time series over certain period of time, resulting into better detection.…”

Section: Machine Learning Based Anomaly Detection Techniquesmentioning

confidence: 99%

DAE : Discriminatory Auto-Encoder for multivariate time-series anomaly detection in air transportation

Chevrot,

Vernotte,

Legeard

2021

Preprint

View full text Add to dashboard Cite

The Automatic Dependent Surveillance-Broadcast protocol is one of the latest compulsory advances in air surveillance. While it supports the tracking of the ever-growing number of aircraft in the air, it also introduces cybersecurity issues that must be mitigated e.g., false data injection attacks where an attacker emits fake surveillance information. The recent data sources and tools available to obtain flight tracking records allow the researchers to create datasets and develop Machine Learning models capable of detecting such anomalies in En-Route trajectories. In this context, we propose a novel multivariate anomaly detection model called Discriminatory Auto-Encoder (DAE). It uses the baseline of a regular LSTM-based auto-encoder but with several decoders, each getting data of a specific flight phase (e.g. climbing, cruising or descending) during its training. To illustrate the DAE's efficiency, an evaluation dataset was created using real-life anomalies as well as realistically crafted ones, with which the DAE as well as three anomaly detection models from the literature were evaluated. Results show that the DAE achieves better results in both accuracy and speed of detection. The dataset, the models implementations and the evaluation results are available in an online repository, thereby enabling replicability and facilitating future experiments.

show abstract

Section: Machine Learning Based Anomaly Detection Techniquesmentioning

confidence: 99%

DAE : Discriminatory Auto-Encoder for multivariate time-series anomaly detection in air transportation

Chevrot,

Vernotte,

Legeard

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…VAEs can be combined with generative adversarial networks (GANs) [16] to enhance the quality of the converted speech, where the decoder of the VAE is shared with the generator of the GAN [17]. The VAE-GAN can be extended to include the cycle-consistency loss [18], [19] to further improve the voice quality, especially for non-parallel training data. This is known as a cycle-consistent variational autoencoding generative adversarial network (CycleVAE-GAN) [19].…”

Section: B Voice Conversionmentioning

confidence: 99%

Speaker Anonymization for Personal Information Protection Using Voice Conversion Techniques

et al. 2020

Self Cite

View full text Add to dashboard Cite

As speech-based user interfaces integrated in the devices such as AI speakers become ubiquitous, a large amount of user voice data is being collected to enhance the accuracy of speech recognition systems. Since such voice data contain personal information that can endanger the privacy of users, the issue of privacy protection in the speech data has garnered increasing attention after the introduction of the General Data Protection Regulation in the EU, which implies that restrictions and safety measures for the use of speech data become essential. This study aims to filter the speaker-related voice biometrics present in speech data such as voice fingerprint without altering the linguistic content to preserve the usefulness of the data while protecting the privacy of users. To achieve this, we propose an algorithm that produces anonymized speeches by adopting many-to-many voice conversion techniques based on variational autoencoders (VAEs) and modifying the speaker identity vectors of the VAE input to anonymize the speech data. We validated the effectiveness of the proposed method by measuring the speaker-related information and the original linguistic information retained in the resultant speech, using an open source speaker recognizer and a deep neural network-based automatic speech recognizer, respectively. Using the proposed method, the speaker identification accuracy of the speech data was reduced to 0.1-9.2%, indicating successful anonymization, while the speech recognition accuracy was maintained as 78.2-81.3%.

show abstract

“…CycleGAN and its variations [67,51] have seen considerable success in performing cross-modal unsupervised domain transfer, for medical imaging [19,56] and audio to visual translation [17], but often encode information as high frequency signal that is invisible to the human eye and susceptible to adversarial attacks [6]. An alternative approach involves training a VAE, subject to a cycle-consistency condition [26,62], but these works were restricted to domain transfers within a single modality. Most similar is the joint audio and video model proposed by Tian et al [55], which uses a VAE to map between two incompatible latent spaces using supervised alignment of attributes, however, it operates on a word not phoneme level, and has no mechanism to ensure temporal smoothness nor information throughput.…”

Section: Related Workmentioning

confidence: 99%

“…However, the learned encoders are only approximations to the true distribution and not all points in Z are modelled equally well, leading to information loss. To bridge different latent space structures, we introduce an additional cycle constraint that works without image correspondence and ensures that samples from the audio latent space posterior are reconstructed well, similar in spirit to [26,62]. Figure 4 shows the cyclic chaining from audio latent code to video and back.…”

Section: Linking Audio and Visual Spacesmentioning

confidence: 99%

AudioViewer: Learning to Visualize Sounds

Zhang¹,

Song²,

Willis³

et al. 2020

Preprint

View full text Add to dashboard Cite

Sensory substitution can help persons with perceptual deficits. In this work, we attempt to visualize audio with video. Our long-term goal is to create sound perception for hearing impaired people, for instance, to facilitate feedback for training deaf speech. Different from existing models that translate between speech and text or text and images, we target an immediate and low-level translation that applies to generic environment sounds and human speech without delay. No canonical mapping is known for this artificial translation task. Our design is to translate from audio to video by compressing both into a common latent space with shared structure. Our core contribution is the development and evaluation of learned mappings that respect human perception limits and maximize user comfort by enforcing priors and combining strategies from unpaired image translation and disentanglement. We demonstrate qualitatively and quantitatively that our AudioViewer model maintains important audio features in the generated video and that generated videos of faces and numbers are well suited for visualizing high-dimensional audio features since they can easily be parsed by humans to match and distinguish between sounds, words, and speakers.

show abstract

Many-to-Many Voice Conversion Using Cycle-Consistent Variational Autoencoder with Multiple Decoders

Cited by 11 publications

References 0 publications

DAE : Discriminatory Auto-Encoder for multivariate time-series anomaly detection in air transportation

DAE : Discriminatory Auto-Encoder for multivariate time-series anomaly detection in air transportation

Speaker Anonymization for Personal Information Protection Using Voice Conversion Techniques

AudioViewer: Learning to Visualize Sounds

Contact Info

Product

Resources

About