Phase reconstruction from amplitude spectrograms based on directional-statistics deep neural networks

Takamichi, Shinnosuke; Saito, Yuki; Takamune, Norihiro; Kitamura, Daichi; Saruwatari, Hiroshi

doi:10.1016/j.sigpro.2019.107368

Cited by 21 publications

(27 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Directional-statistics DNN [27,28] Quantizing approach [13,14] Griffin-Lim algorithm [20] Fast Griffin-Lim algorithm [21] Consistency-based approach Fig. 1.…”

Section: Dnn-based Approachmentioning

confidence: 99%

“…DNNs can automatically discover the structures of signals in the training dataset and utilize the obtained knowledge for phase reconstruction. This approach has achieved promising results in speech synthesis [27], [28], and it has a high potential for further improvements in various applications because DNNs can acquire the knowledge of any signals [42]. The application of the approach is not restricted to specially structured signals like a sum of sinusoids.…”

Section: Dnn-based Approachmentioning

confidence: 99%

“…This bias directly affects the estimated residual component in DeGLI. In contrast, the gating mechanism given in (28) preserves the phase because the sigmoid function takes a real number within 0 to 1. Thus, the proposed nonlinear function does not induce such bias to the estimated residual.…”

Section: Proposed Complex Dnnmentioning

confidence: 99%

See 2 more Smart Citations

Deep Griffin–Lim Iteration: Trainable Iterative Phase Reconstruction Using Neural Network

Masuyama

Yatabe

Oikawa

et al. 2021

IEEE J. Sel. Top. Signal Process.

View full text Add to dashboard Cite

In this paper, we propose a phase reconstruction framework, named Deep Griffin-Lim Iteration (DeGLI). Phase reconstruction is a fundamental technique for improving the quality of sound obtained through some process in the timefrequency domain. It has been shown that the recent methods using deep neural networks (DNN) outperformed the conventional iterative phase reconstruction methods such as the Griffin-Lim algorithm (GLA). However, the computational cost of DNNbased methods is not adjustable at the time of inference, which may limit the range of applications. To address this problem, we combine the iterative structure of GLA with a DNN so that the computational cost becomes adjustable by changing the number of iterations of the proposed DNN-based component. A training method that is independent of the number of iterations for inference is also proposed to minimize the computational cost of the training. This training method, named sub-block training by denoising (SBTD), avoids recursive use of the DNN and enables training of DeGLI with a single sub-block (corresponding to one GLA iteration). Furthermore, we propose a complex DNN based on complex convolution layers with gated mechanisms and investigated its performance in terms of the proposed framework. Through several experiments, we found that DeGLI significantly improved both objective and subjective measures from GLA by incorporating the DNN, and its sound quality was comparable to those of neural vocoders.

show abstract

“…Directional-statistics DNN [27,28] Quantizing approach [13,14] Griffin-Lim algorithm [20] Fast Griffin-Lim algorithm [21] Consistency-based approach Fig. 1.…”

Section: Dnn-based Approachmentioning

confidence: 99%

Section: Dnn-based Approachmentioning

confidence: 99%

See 1 more Smart Citation

Deep Griffin–Lim Iteration: Trainable Iterative Phase Reconstruction Using Neural Network

Masuyama

Yatabe

Oikawa

et al. 2021

IEEE J. Sel. Top. Signal Process.

View full text Add to dashboard Cite

show abstract

“…Whereas the lifter of the minimum-phase filter is fixed, that of our method is trained from speech data to determine the phases of a truncated filter. Our lifter-training method can be viewed as a framework of DNN-based phase reconstruction from the amplitude spectrum [15]. Second, for fullband VC, we also propose a frequency-band-wise modeling method based on sub-band multi-rate signal processing (hereafter, "sub-band modeling method") [16].…”

Section: Introductionmentioning

confidence: 99%

Real-Time Full-Band Voice Conversion with Sub-Band Modeling and Data-Driven Phase Estimation of Spectral Differentials

Saeki

Saito

Takamichi

et al. 2021

IEICE Trans. Inf. & Syst.

Self Cite

View full text Add to dashboard Cite

This paper proposes two high-fidelity and computationally efficient neural voice conversion (VC) methods based on a direct waveform modification using spectral differentials. The conventional spectraldifferential VC method with a minimum-phase filter achieves high-quality conversion for narrow-band (16 kHz-sampled) VC but requires heavy computational cost in filtering. This is because the minimum phase obtained using a fixed lifter of the Hilbert transform often results in a long-tap filter. Furthermore, when we extend the method to full-band (48 kHz-sampled) VC, the computational cost is heavy due to increased sampling points, and the converted-speech quality degrades due to large fluctuations in the high-frequency band. To construct a short-tap filter, we propose a liftertraining method for data-driven phase reconstruction that trains a lifter of the Hilbert transform by taking into account filter truncation. We also propose a frequency-band-wise modeling method based on sub-band multirate signal processing (sub-band modeling method) for full-band VC. It enhances the computational efficiency by reducing sampling points of signals converted with filtering and improves converted-speech quality by modeling only the low-frequency band. We conducted several objective and subjective evaluations to investigate the effectiveness of the proposed methods through implementation of the real-time, online, full-band VC system we developed, which is based on the proposed methods. The results indicate that 1) the proposed lifter-training method for narrow-band VC can shorten the tap length to 1/16 without degrading the converted-speech quality, and 2) the proposed sub-band modeling method for full-band VC can improve the converted-speech quality while reducing the computational cost, and 3) our real-time, online, full-band VC system can convert 48 kHz-sampled speech in real time attaining the converted speech with a 3.6 out of 5.0 mean opinion score of naturalness.

show abstract

“…Moreover, besides issues of computational complexity the above methods are mostly applied for the detection of a single target, while general solutions should also support efficient detection of multiple targets (see References [12,13] for recent approaches tackling this issue). A promising emerging technology to cope with these limitations is represented by deep learning [14], which has recently achieved state-of-the-art performance in a variety of difficult pattern recognition tasks, ranging from image classification [15] to speech recognition [16,17], without requiring domain-specific expert knowledge about the signal characteristics.…”

Section: Introductionmentioning

confidence: 99%

Combining Denoising Autoencoders and Dynamic Programming for Acoustic Detection and Tracking of Underwater Moving Targets

Testolin

Diamant

2020

Sensors

View full text Add to dashboard Cite

Accurate detection and tracking of moving targets in underwater environments pose significant challenges, because noise in acoustic measurements (e.g., SONAR) makes the signal highly stochastic. In continuous marine monitoring a further challenge is related to the computational complexity of the signal processing pipeline—due to energy constraints, in off-shore monitoring platforms algorithms should operate in real time with limited power consumption. In this paper, we present an innovative method that allows to accurately detect and track underwater moving targets from the reflections of an active acoustic emitter. Our system is based on a computationally- and energy-efficient pre-processing stage carried out using a deep convolutional denoising autoencoder (CDA), whose output is then fed to a probabilistic tracking method based on the Viterbi algorithm. The CDA is trained on a large database of more than 20,000 reflection patterns collected during 50 designated sea experiments. System performance is then evaluated on a controlled dataset, for which ground truth information is known, as well as on recordings collected during different sea experiments. Results show that, compared to the benchmark, our method achieves a favorable trade-off between detection and false alarm rate, as well as improved tracking accuracy.

show abstract

Phase reconstruction from amplitude spectrograms based on directional-statistics deep neural networks

Cited by 21 publications

References 7 publications

Deep Griffin–Lim Iteration: Trainable Iterative Phase Reconstruction Using Neural Network

Deep Griffin–Lim Iteration: Trainable Iterative Phase Reconstruction Using Neural Network

Real-Time Full-Band Voice Conversion with Sub-Band Modeling and Data-Driven Phase Estimation of Spectral Differentials

Combining Denoising Autoencoders and Dynamic Programming for Acoustic Detection and Tracking of Underwater Moving Targets

Contact Info

Product

Resources

About