Time-Frequency Networks for Audio Super-Resolution

Lim, Teck Yian; Yeh, Raymond A.; Xu, Yijia; Minh, N.; Hasegawa‐Johnson, Mark

doi:10.1109/icassp.2018.8462049

Cited by 62 publications

(36 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Li et al [28] adapted an EnvNet structure to approximate the spectral feature extraction from waveform. Time-Frequency Networks [29] use dual branches with a spectral fusion layer to combine the information. In addition, time-frequency losses have been widely adopted by time-domain methods to encourage matching of the upperband spectral energy [28,30].…”

Section: Related Workmentioning

confidence: 99%

Bandwidth Extension is All You Need

Wang

Finkelstein

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Speech generation and enhancement have seen recent breakthroughs in quality thanks to deep learning. These methods typically operate at a limited sampling rate of 16-22kHz due to computational complexity and available datasets. This limitation imposes a gap between the output of such methods and that of high-fidelity (≥44kHz) real-world audio applications. This paper proposes a new bandwidth extension (BWE) method that expands 8-16kHz speech signals to 48kHz. The method is based on a feed-forward WaveNet architecture trained with a GAN-based deep feature loss. A mean-opinionscore (MOS) experiment shows significant improvement in quality over state-of-the-art BWE methods. An AB test reveals that our 16to-48kHz BWE is able to achieve fidelity that is typically indistinguishable from real high-fidelity recordings. We use our method to enhance the output of recent speech generation and denoising methods, and experiments demonstrate significant improvement in sound quality over these baselines. We propose this as a general approach to narrow the gap between generated speech and recorded speech, without the need to adapt such methods to higher sampling rates.

show abstract

Section: Related Workmentioning

confidence: 99%

Bandwidth Extension is All You Need

Wang

Finkelstein

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…In 2018, Lim et al [20] proposed a novel deep neural network structure to perform audio super-resolution in time-frequency domain. Generally, promising superresolution results can be achieved in the computer audio/ vision fields by employing data-driven super-resolution techniques.…”

Section: Related Workmentioning

confidence: 99%

Well log data super-resolution based on locally linear embedding

Han

Gao

Cao

et al. 2021

Oil Gas Sci. Technol. – Rev. IFP Energies nouvelles

View full text Add to dashboard Cite

Unconventional remaining oil and gas resources such as tight oil, shale oil, and coalbed gas are currently the focus of the exploration and development of major oil fields all over the world. Therefore, to make best understand of target reservoirs, enhancing the vertical resolution of well log data is crucial important. However, in the face of the continuous low-level fluctuations of international oil price, large scale use of expensive high resolution well logging hardware tools has always been unaffordable and unacceptable. In another aspect, traditional well log interpolation methods can always not realize high reliable information enhancement for crucial high frequency components. In this paper, in order to improve the well log data super-resolution performance, we propose for the first time to employ Locally Linear Embedding (LLE) technique to reveal the nonlinear mapping relationship between 2-times-scale-difference well log data. Several super resolution experiments with well log data from a given area of Daqing Oil field, China, were conducted. Experimental results illustrated that the proposed LLE-based method can efficiently achieve more reliable super-resolution results than other state-of-the-art methods.

show abstract

“…In [28], gray-box modeling is proposed for nonlinear effects with long temporal dependencies such as compressors. The architecture is based on U-Net [47] and Time-Frequency [48] networks, where using input-output measurements and knowledge of the attack and release gate times are used to emulate different compressors and their respective controls. Similarly, RNNs for real-time black-box modeling of tube amplifiers and distortion pedals were explored in [23] and static configurations of tube amplifiers in [21,22].…”

Section: Deep Learning For Audio Effects Modelingmentioning

confidence: 99%

Deep Learning for Black-Box Modeling of Audio Effects

2020

View full text Add to dashboard Cite

Virtual analog modeling of audio effects consists of emulating the sound of an audio processor reference device. This digital simulation is normally done by designing mathematical models of these systems. It is often difficult because it seeks to accurately model all components within the effect unit, which usually contains various nonlinearities and time-varying components. Most existing methods for audio effects modeling are either simplified or optimized to a very specific circuit or type of audio effect and cannot be efficiently translated to other types of audio effects. Recently, deep neural networks have been explored as black-box modeling strategies to solve this task, i.e., by using only input–output measurements. We analyse different state-of-the-art deep learning models based on convolutional and recurrent neural networks, feedforward WaveNet architectures and we also introduce a new model based on the combination of the aforementioned models. Through objective perceptual-based metrics and subjective listening tests we explore the performance of these models when modeling various analog audio effects. Thus, we show virtual analog models of nonlinear effects, such as a tube preamplifier; nonlinear effects with memory, such as a transistor-based limiter and nonlinear time-varying effects, such as the rotating horn and rotating woofer of a Leslie speaker cabinet.

show abstract

Time-Frequency Networks for Audio Super-Resolution

Cited by 62 publications

References 7 publications

Bandwidth Extension is All You Need

Bandwidth Extension is All You Need

Well log data super-resolution based on locally linear embedding

Deep Learning for Black-Box Modeling of Audio Effects

Contact Info

Product

Resources

About