Deep Learning-Based Estimation of Reverberant Environment for Audio Data Augmentation

Yun, Deokgyu; Choi, Seung Ho

doi:10.3390/s22020592

Cited by 7 publications

(3 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, there are augmentation methods that are applied to the waveform of a signal. These include changing the pitch [37], increasing/decreasing the volume of the signal, changing the speed (speeding up or slowing down the original signal), adding random noise, silence, mixing the signal with background sounds from different types of acoustic scenes [38], adding reverb [39,40], time shift (right or left), etc.…”

Section: Related Workmentioning

confidence: 99%

Generating Synthetic Sperm Whale Voice Data Using StyleGAN2-ADA

Kopets,

Shpilevaya,

Vasilchenko

et al. 2024

BDCC

View full text Add to dashboard Cite

The application of deep learning neural networks enables the processing of extensive volumes of data and often requires dense datasets. In certain domains, researchers encounter challenges related to the scarcity of training data, particularly in marine biology. In addition, many sounds produced by sea mammals are of interest in technical applications, e.g., underwater communication or sonar construction. Thus, generating synthetic biological sounds is an important task for understanding and studying the behavior of various animal species, especially large sea mammals, which demonstrate complex social behavior and can use hydrolocation to navigate underwater. This study is devoted to generating sperm whale vocalizations using a limited sperm whale click dataset. Our approach utilizes an augmentation technique predicated on the transformation of audio sample spectrograms, followed by the employment of the generative adversarial network StyleGAN2-ADA to generate new audio data. The results show that using the chosen augmentation method, namely mixing along the time axis, makes it possible to create fairly similar clicks of sperm whales with a maximum deviation of 2%. The generation of new clicks was reproduced on datasets using selected augmentation approaches with two neural networks: StyleGAN2-ADA and WaveGan. StyleGAN2-ADA, trained on an augmented dataset using the axis mixing approach, showed better results compared to WaveGAN.

show abstract

Section: Related Workmentioning

confidence: 99%

Generating Synthetic Sperm Whale Voice Data Using StyleGAN2-ADA

Kopets,

Shpilevaya,

Vasilchenko

et al. 2024

BDCC

View full text Add to dashboard Cite

show abstract

“…DL systems, especially those based on CNNs, have shown to be promising in various identification and classification tasks, most notably in image recognition [6,7]. Actually, the application field can be extended to sound-event classification after the basic sound information is converted and transformed to the resulting image form representations (such as spectrograms and scalograms) [8,9]. In addition, the creation of transfer-learning models using pre-trained weights, discriminative learning, and fine-tuning concepts has been proven.…”

Section: Introductionmentioning

confidence: 99%

Environmental Sound Classification Based on Transfer-Learning Techniques with Multiple Optimizers

et al. 2022

View full text Add to dashboard Cite

The last decade has seen increased interest in environmental sound classification (ESC) due to the increased complexity and rich information of ambient sounds. The state-of-the-art methods for ESC are based on transfer learning paradigms that often utilize learned representations from common image-classification problems. This paper aims to determine the effectiveness of employing pre-trained convolutional neural networks (CNNs) for audio categorization and the feasibility of retraining. This study investigated various hyper-parameters and optimizers, such as optimal learning rate, epochs, and Adam, Adamax, and RMSprop optimizers for several pre-trained models, such as Inception, and VGG, ResNet, etc. Firstly, the raw sound signals were transferred into an image format (log-Mel spectrogram). Then, the selected pre-trained models were applied to the obtained spectrogram data. In addition, the effect of essential retraining factors on classification accuracy and processing time was investigated during CNN training. Various optimizers (such as Adam, Adamax, and RMSprop) and hyperparameters were utilized for evaluating the proposed method on the publicly accessible sound dataset UrbanSound8K. The proposed method achieves 97.25% and 95.5% accuracy on the provided dataset using the pre-trained DenseNet201 and the ResNet50V2 CNN models, respectively.

show abstract

“…Recent data augmentation approaches deviate from conventional methods. One of the possible cases of data augmentation is employing a neural-network-based approach [17]. Convolutional neural networks are applied as data augmentation tools for speech data.…”

Section: Introductionmentioning

confidence: 99%

Fractional-Order Calculus-Based Data Augmentation Methods for Environmental Sound Classification with Deep Learning

Yazgac

Kırcı

2022

Fractal Fract

View full text Add to dashboard Cite

In this paper, we propose two fractional-order calculus-based data augmentation methods for audio signals. The first approach is based on fractional differentiation of the Mel scale. By using a randomly selected fractional derivation order, we are warping the Mel scale, therefore, we aim to augment Mel-scale-based time-frequency representations of audio data. The second approach is based on previous fractional-order image edge enhancement methods. Since multiple deep learning approaches treat Mel spectrogram representations like images, a fractional-order differential-based mask is employed. The mask parameters are produced with respect to randomly selected fractional-order derivative parameters. The proposed data augmentation methods are applied to the UrbanSound8k environmental sound dataset. For the classification of the dataset and testing the methods, an arbitrary convolutional neural network is implemented. Our results show that fractional-order calculus-based methods can be employed as data augmentation methods. Increasing the dataset size to six times the original size, the classification accuracy result increased by around 8.5%. Additional tests on more complex networks also produced better accuracy results compared to a non-augmented dataset. To our knowledge, this paper is the first example of employing fractional-order calculus as an audio data augmentation tool.

show abstract

Deep Learning-Based Estimation of Reverberant Environment for Audio Data Augmentation

Cited by 7 publications

References 16 publications

Generating Synthetic Sperm Whale Voice Data Using StyleGAN2-ADA

Generating Synthetic Sperm Whale Voice Data Using StyleGAN2-ADA

Environmental Sound Classification Based on Transfer-Learning Techniques with Multiple Optimizers

Fractional-Order Calculus-Based Data Augmentation Methods for Environmental Sound Classification with Deep Learning

Contact Info

Product

Resources

About