Improving Deep Learning with Generic Data Augmentation

Taylor, Luke A.; Nitschke, Geoff

doi:10.1109/ssci.2018.8628742

Cited by 523 publications

(342 citation statements)

References 6 publications

Supporting

Mentioning

334

Contrasting

Unclassified

Order By: Relevance

“…We note that there are countless sketching protocols to represent molecules, and hence, future benchmarking studies will be needed to thoroughly examine their predictive signal. Similarly, elucidating the most convenient strategies to perform data augmentation is an area of intense research [99][100][101], also for chemical structure-activity modelling [82,102]. In the case of ConvNets, multiple representations of the same molecules generated using diverse sketching schemes (e.g., using diverse SMILES encoding rules [102]) might be implemented to perform data augmentation.…”

Section: Resultsmentioning

confidence: 99%

KekuleScope: prediction of cancer cell line sensitivity and compound potency using convolutional neural networks trained on compound images

Cortés-Ciriano

Bender

2019

J Cheminform

View full text Add to dashboard Cite

The application of convolutional neural networks (ConvNets) to harness high-content screening images or 2D compound representations is gaining increasing attention in drug discovery. However, existing applications often require large data sets for training, or sophisticated pretraining schemes. Here, we show using 33 IC 50 data sets from ChEMBL 23 that the in vitro activity of compounds on cancer cell lines and protein targets can be accurately predicted on a continuous scale from their Kekulé structure representations alone by extending existing architectures (AlexNet, DenseNet-201, ResNet152 and VGG-19), which were pretrained on unrelated image data sets. We show that the predictive power of the generated models, which just require standard 2D compound representations as input, is comparable to that of Random Forest (RF) models and fully-connected Deep Neural Networks trained on circular (Morgan) fingerprints. Notably, including additional fully-connected layers further increases the predictive power of the ConvNets by up to 10%. Analysis of the predictions generated by RF models and ConvNets shows that by simply averaging the output of the RF models and ConvNets we obtain significantly lower errors in prediction for multiple data sets, although the effect size is small, than those obtained with either model alone, indicating that the features extracted by the convolutional layers of the ConvNets provide complementary predictive signal to Morgan fingerprints. Lastly, we show that multi-task ConvNets trained on compound images permit to model COX isoform selectivity on a continuous scale with errors in prediction comparable to the uncertainty of the data. Overall, in this work we present a set of ConvNet architectures for the prediction of compound activity from their Kekulé structure representations with state-of-the-art performance, that require no generation of compound descriptors or use of sophisticated image processing techniques. The code needed to reproduce the results presented in this study and all the data sets are provided at https://github.com/isidroc/kekulescope .

show abstract

Section: Resultsmentioning

confidence: 99%

KekuleScope: prediction of cancer cell line sensitivity and compound potency using convolutional neural networks trained on compound images

Cortés-Ciriano

Bender

2019

J Cheminform

View full text Add to dashboard Cite

show abstract

“…Unfortunately, generating several realizations of the geological model with standard geostatistical algorithms may be very challenging. One possible solution we intend to investigate in the future is to use data augmentation (Yaeger et al, 1997;Taylor and Nitschke, 2017) and transfer learning techniques (Hoo-Chang et al, 2018;Cheng and Malhi, 2017). Data augmentation consist of a series of affine transformations applied to the input data to increase the training set.…”

Section: Commentsmentioning

confidence: 99%

Towards a robust parameterization for conditioning facies models using deep variational autoencoders and ensemble smoother

Canchumuni

Emerick

Pacheco

2019

Computers & Geosciences

122

View full text Add to dashboard Cite

History matching is a jargon used to refer to the data assimilation problem in oil and gas reservoirs. The literature about history matching is vast and despite the impressive number of methods proposed and the significant progresses reported in the last decade, conditioning reservoir models to dynamic data is still a challenging task. Ensemble-based methods are among the most successful and efficient techniques currently available for history matching. These methods are usually able to achieve reasonable data matches, especially if an iterative formulation is employed. However, they sometimes fail to preserve the geological realism of the model, which is particularly evident in reservoir with complex facies distributions. This occurs mainly because of the Gaussian assumptions inherent in these methods. This fact has encouraged an intense research activity to develop parameterizations for facies history matching. Despite the large number of publications, the development of robust parameterizations for facies remains an open problem.Deep learning techniques have been delivering impressive results in a number of different areas and the first applications in data assimilation in geoscience have started to appear in literature. The present paper reports the current results of our investigations on the use of deep neural networks towards the construction of a continuous parameterization of facies which can be used for data assimilation with ensemble methods. Specifically, we use a convolutional variational autoencoder and the ensemble smoother with multiple data assimilation. We tested the parameterization in three synthetic history-matching problems with channelized facies. We focus on this type of facies because they are among the most challenging to preserve after the assimilation of data. The parameterization showed promising results outperforming previous methods and generating well-defined channelized facies. However, more research is still required before deploying these methods for operational use.

show abstract

“…1(c). Although both rotation and flip augmentation methods achieve similar accuracy improvements for image classification [24], [25], it is an open question about which one is preferred for radio modulation classification. After the Gaussian noise augmentation, the image is full of 'snow' and the received radio symbols are deviated as shown in Fig.…”

Section: Introductionmentioning

confidence: 99%

Data Augmentation for Deep Learning-Based Radio Modulation Classification

et al. 2020

View full text Add to dashboard Cite

Deep learning has recently been applied to automatically classify the modulation categories of received radio signals without manual experience. However, training deep learning models requires massive volume of data. An insufficient training data will cause serious overfitting problem and degrade the classification accuracy. To cope with small dataset, data augmentation has been widely used in image processing to expand the dataset and improve the robustness of deep learning models. However, in wireless communication areas, the effect of different data augmentation methods on radio modulation classification has not been studied yet. In this paper, we evaluate different data augmentation methods via a state-ofthe-art deep learning-based modulation classifier. Based on the characteristics of modulated signals, three augmentation methods are considered, i.e., rotation, flip, and Gaussian noise, which can be applied in both training phase and inference phase of the deep learning algorithm. Numerical results show that all three augmentation methods can improve the classification accuracy. Among which, the rotation augmentation method outperforms the flip method, both of which achieve higher classification accuracy than the Gaussian noise method. Given only 12.5% of training dataset, a joint rotation and flip augmentation policy can achieve even higher classification accuracy than the baseline with initial 100% training dataset without augmentation. Furthermore, with data augmentation, radio modulation categories can be successfully classified using shorter radio samples, leading to a simplified deep learning model and shorter the classification response time.

show abstract

Improving Deep Learning with Generic Data Augmentation

Cited by 523 publications

References 6 publications

KekuleScope: prediction of cancer cell line sensitivity and compound potency using convolutional neural networks trained on compound images

KekuleScope: prediction of cancer cell line sensitivity and compound potency using convolutional neural networks trained on compound images

Towards a robust parameterization for conditioning facies models using deep variational autoencoders and ensemble smoother

Data Augmentation for Deep Learning-Based Radio Modulation Classification

Contact Info

Product

Resources

About