Coarse-to-fine Optimization for Speech Enhancement

Yao, Jian; Al-Dahle, Ahmad

doi:10.48550/arxiv.1908.08044

Cited by 5 publications

(8 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To evaluate MASnet, we use a standard dataset [29,30] and compare with LLASnet-8, LLASnet-15 and the CRMRN U-net from [9,8]. All of our experiments are implemented with Py-Torch [31].…”

Section: Methodsmentioning

confidence: 99%

“…Spectrogram-based speech enhancement methods often treat STFT representations as images [9,8]. This is fine at trainingtime, but at test-time and especially in low-latency applications such as telephony it is important to be able to output processed audio immediately, without having to record several seconds that could be transformed into a spectrogram.…”

Section: Low-latency Speech Enhancementmentioning

confidence: 99%

“…The first imporant choice is that of signal representation. Some algorithms work directly with waveforms [3, 4, 5, 6, 7], while others are based on spectrogram representations computed with the short-time Fourier transform (STFT) [8,9]. Some other methods work with Mel-scale spectrograms [1].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Efficient Low-Latency Speech Enhancement with Mobile Audio Streaming Networks

Romaniuk¹,

Masztalski²,

Piaskowski³

et al. 2020

Interspeech 2020

View full text Add to dashboard Cite

We propose Mobile Audio Streaming Networks (MASnet) for efficient low-latency speech enhancement, which is particularly suitable for mobile devices and other applications where computational capacity is a limitation. MASnet processes linearscale spectrograms, transforming successive noisy frames into complex-valued ratio masks which are then applied to the respective noisy frames. MASnet can operate in a low-latency incremental inference mode which matches the complexity of layer-by-layer batch mode. Compared to a similar fullyconvolutional architecture, MASnet incorporates depthwise and pointwise convolutions for a large reduction in fused multiplyaccumulate operations per second (FMA/s), at the cost of some reduction in SNR.

show abstract

“…To evaluate MASnet, we use a standard dataset [29,30] and compare with LLASnet-8, LLASnet-15 and the CRMRN U-net from [9,8]. All of our experiments are implemented with Py-Torch [31].…”

Section: Methodsmentioning

confidence: 99%

Section: Low-latency Speech Enhancementmentioning

confidence: 99%

See 1 more Smart Citation

Efficient Low-Latency Speech Enhancement with Mobile Audio Streaming Networks

Romaniuk¹,

Masztalski²,

Piaskowski³

et al. 2020

Interspeech 2020

View full text Add to dashboard Cite

show abstract

“…The idea of solving tasks in a coarse-to-fine order has previously been explored in computer vision and signal processing [e.g., for object detection and recognition 14,3,38,45], head pose estimation [62], or more general computer vision tasks [32,36,47,65,70]. Similarly, coarse-to-fine ideas have also been used for various tasks in natural language processing [e.g., 30,12,68]. Our approach is different in that it is widely applicable; it does not depend on the problem space at hand, but can rather be applied as-is to any classification problem.…”

Section: Related Workmentioning

confidence: 99%

Coarse-to-Fine Curriculum Learning

Stretcu,

Platanios,

Mitchell

et al. 2021

Preprint

View full text Add to dashboard Cite

When faced with learning challenging new tasks, humans often follow sequences of steps that allow them to incrementally build up the necessary skills for performing these new tasks. However, in machine learning, models are most often trained to solve the target tasks directly. Inspired by human learning, we propose a novel curriculum learning approach which decomposes challenging tasks into sequences of easier intermediate goals that are used to pre-train a model before tackling the target task. We focus on classification tasks, and design the intermediate tasks using an automatically constructed label hierarchy. We train the model at each level of the hierarchy, from coarse labels to fine labels, transferring acquired knowledge across these levels. For instance, the model will first learn to distinguish animals from objects, and then use this acquired knowledge when learning to classify among more fine-grained classes such as cat, dog, car, and truck. Most existing curriculum learning algorithms for supervised learning consist of scheduling the order in which the training examples are presented to the model. In contrast, our approach focuses on the output space of the model. We evaluate our method on several established datasets and show significant performance gains especially on classification problems with many labels. We also evaluate on a new synthetic dataset which allows us to study multiple aspects of our method.

show abstract

“…Second, we propose a new time-domain loss function, an emphasized multi-scale cos similarity loss function. A time-domain loss function has recently been used as a popular loss function [6,7,8,9,10]. To better design the time-domain cos similarity loss function proposed in [6], we change it into a multi-scale version of it with proper emphasis functions and show the effectiveness of it.…”

Section: Introductionmentioning

confidence: 99%

Phase-aware Single-stage Speech Denoising and Dereverberation with U-Net

Choi¹,

Heo²,

Lee³

et al. 2020

Preprint

View full text Add to dashboard Cite

In this work, we tackle a denoising and dereverberation problem with a single-stage framework. Although denoising and dereverberation may be considered two separate challenging tasks, and thus, two modules are typically required for each task, we show that a single deep network can be shared to solve the two problems. To this end, we propose a new masking method called phase-aware β-sigmoid mask (PHM), which reuses the estimated magnitude values to estimate the clean phase by respecting the triangle inequality in the complex domain between three signal components such as mixture, source and the rest. Two PHMs are used to deal with direct and reverberant source, which allows controlling the proportion of reverberation in the enhanced speech at inference time. In addition, to improve the speech enhancement performance, we propose a new time-domain loss function and show a reasonable performance gain compared to MSE loss in the complex domain. Finally, to achieve a real-time inference, an optimization strategy for U-Net is proposed which significantly reduces the computational overhead up to 88.9% compared to the naïve version 1 .

show abstract

Coarse-to-fine Optimization for Speech Enhancement

Cited by 5 publications

References 27 publications

Efficient Low-Latency Speech Enhancement with Mobile Audio Streaming Networks

Efficient Low-Latency Speech Enhancement with Mobile Audio Streaming Networks

Coarse-to-Fine Curriculum Learning

Phase-aware Single-stage Speech Denoising and Dereverberation with U-Net

Contact Info

Product

Resources

About