Adversarial Training for Speech Super-Resolution

Eskimez, Şefik Emre; Duan, Zhiyao

doi:10.1109/jstsp.2019.2909077

Cited by 27 publications

(29 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The super-resolution problem asks us to recover unobserved data x given a down-sampled observation y = g(x). For 1-dimensional (audio) super-resolution, x ∈ R n , y ∈ R n/r , and y i = x ri (Kuleshov et al, 2017;Eskimez et al, 2019).…”

Section: Super-resolutionmentioning

confidence: 99%

Parallel and Flexible Sampling from Autoregressive Models via Langevin Dynamics

Jayaram¹,

Thickstun²

2021

Preprint

View full text Add to dashboard Cite

This paper introduces an alternative approach to sampling from autoregressive models. Autoregressive models are typically sampled sequentially, according to the transition dynamics defined by the model. Instead, we propose a sampling procedure that initializes a sequence with white noise and follows a Markov chain defined by Langevin dynamics on the global loglikelihood of the sequence. This approach parallelizes the sampling process and generalizes to conditional sampling. Using an autoregressive model as a Bayesian prior, we can steer the output of a generative model using a conditional likelihood or constraints. We apply these techniques to autoregressive models in the visual and audio domains, with competitive results for audio source separation, super-resolution, and inpainting.

show abstract

Section: Super-resolutionmentioning

confidence: 99%

Parallel and Flexible Sampling from Autoregressive Models via Langevin Dynamics

Jayaram¹,

Thickstun²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…For this problem, a line of early works have developed inference algorithms on a set of probabilistic models e.g., Gaussian mixture models [1], hidden Markov models [2] and linear predictive coding [3]. The performance of audio super resolution has been improved remarkably with the recent advances of deep learning techniques, including but not limited to U-Net architecture [4], Generative Adversarial Networks * equal contribution (GANs) [5,6], and normalizing flows [7]. The existing deep learning methods however focus on the fixed scale super resolution, in which neural network is used to have discrete representation of audio signal only at a given set of coordinates corresponding to the target resolution.…”

Section: Introductionmentioning

confidence: 99%

Learning Continuous Representation of Audio for Arbitrary Scale Super Resolution

Kim¹,

Yunjoo²,

Hong³

et al. 2021

Preprint

View full text Add to dashboard Cite

Audio super resolution aims to predict the missing high resolution components of the low resolution audio signals. While audio in nature is continuous signal, current approaches treat it as discrete data (i.e., input is defined on discrete time domain), and consider the super resolution over fixed scale factor (i.e., it is required to train a new neural network to change output resolution). To obtain a continuous representation of audio and enable super resolution for arbitrary scale factor, we propose a method of neural implicit representation, coined Local Implicit representation for Super resolution of Arbitrary scale (LISA). Our method locally parameterizes a chunk of audio as a function of continuous time, and represents each chunk with the local latent codes of neighboring chunks so that the function can extrapolate the signal at any time coordinate, i.e., infinite resolution. To learn a continuous representation for audio, we design a self-supervised learning strategy to practice super resolution tasks up to the original resolution by stochastic selection. Our numerical evaluation shows that LISA outperforms the previous fixed-scale methods with a fraction of parameters, but also is capable of arbitrary scale super resolution even beyond the resolution of training data.

show abstract

“…Simulating adversarial noise and directly using it for training the final classifier (a.k.a. adversarial training [12]) has been investigated in multiple settings (see e.g., [13], [14], [15]), including settings where adversarial noise is used to model worst case scenarios for natural noise (see e.g., [16], [17]).…”

Section: Introductionmentioning

confidence: 99%

Ensemble Noise Simulation to Handle Uncertainty about Gradient-based Adversarial Attacks

Mahfuz,

Sahay,

Gamal

2020

Preprint

View full text Add to dashboard Cite

Gradient-based adversarial attacks on neural networks can be crafted in a variety of ways by varying either how the attack algorithm relies on the gradient, the network architecture used for crafting the attack, or both. Most recent work has focused on defending classifiers in a case where there is no uncertainty about the attacker's behavior (i.e., the attacker is expected to generate a specific attack using a specific network architecture). However, if the attacker is not guaranteed to behave in a certain way, the literature lacks methods in devising a strategic defense. We fill this gap by simulating the attacker's noisy perturbation using a variety of attack algorithms based on gradients of various classifiers. We perform our analysis using a pre-processing Denoising Autoencoder (DAE) defense that is trained with the simulated noise. We demonstrate significant improvements in post-attack accuracy, using our proposed ensemble-trained defense, compared to a situation where no effort is made to handle uncertainty.

show abstract

Adversarial Training for Speech Super-Resolution

Cited by 27 publications

References 20 publications

Parallel and Flexible Sampling from Autoregressive Models via Langevin Dynamics

Parallel and Flexible Sampling from Autoregressive Models via Langevin Dynamics

Learning Continuous Representation of Audio for Arbitrary Scale Super Resolution

Ensemble Noise Simulation to Handle Uncertainty about Gradient-based Adversarial Attacks

Contact Info

Product

Resources

About