Speeding up the Hyperparameter Optimization of Deep Convolutional Neural Networks

Hinz, Tobias; Navarro-Guerrero, Nicolás; Magg, Sven; Wermter, Stefan

doi:10.1142/s1469026818500086

Cited by 102 publications

(53 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Methods for details). Tuning hyperparameters in deep neural networks, especially in complex models such as GANs, can be computationally intensive [60], [61]. Thus, it is quite common in deep learning research to perform one-fold cross-validation [30], [35] or even directly adopt hyperparameter selection from published work [24], [28], [29], [38], [48], [62].…”

Section: Network Trainingmentioning

confidence: 99%

Image Synthesis in Multi-Contrast MRI With Conditional Generative Adversarial Networks

Dar

Yurt

Karacan

et al. 2019

IEEE Trans. Med. Imaging

402

281

View full text Add to dashboard Cite

Acquiring images of the same anatomy with multiple different contrasts increases the diversity of diagnostic information available in an MR exam. Yet, the scan time limitations may prohibit the acquisition of certain contrasts, and some contrasts may be corrupted by noise and artifacts. In such cases, the ability to synthesize unacquired or corrupted contrasts can improve diagnostic utility. For multi-contrast synthesis, the current methods learn a nonlinear intensity transformation between the source and target images, either via nonlinear regression or deterministic neural networks. These methods can, in turn, suffer from the loss of structural details in synthesized images. Here, in this paper, we propose a new approach for multi-contrast MRI synthesis based on conditional generative adversarial networks. The proposed approach preserves intermediate-to-high frequency details via an adversarial loss, and it offers enhanced synthesis performance via pixel-wise and perceptual losses for registered multi-contrast images and a cycle-consistency loss for unregistered images. Information from neighboring cross-sections are utilized to further improve synthesis quality. Demonstrations on T 1-and T 2-weighted images from healthy subjects and patients clearly indicate the superior performance of the proposed approach compared to the previous state-of-the-art methods. Our synthesis approach can help improve the quality and versatility Manuscript

show abstract

Section: Network Trainingmentioning

confidence: 99%

Image Synthesis in Multi-Contrast MRI With Conditional Generative Adversarial Networks

Dar

Yurt

Karacan

et al. 2019

IEEE Trans. Med. Imaging

402

281

View full text Add to dashboard Cite

show abstract

“…In contrast, Bayesian optimization selects the next sampled hyperparameters based on previous evaluations. This has proven more efficient in terms of balancing exploration-8 H. Soto & B. Schurr exploitation of the search space, time consumption, and model performance results, compared to random search (Bergstra et al 2013a;Hinz et al 2018).…”

Section: Hyperparameter Optimization Of Adaptive Neural Networkmentioning

confidence: 99%

DeepPhasePick: A method for Detecting and Picking Seismic Phases from Local Earthquakes based on highly optimized Convolutional and Recurrent Deep Neural Networks

Soto¹,

Schurr²

2020

Preprint

View full text Add to dashboard Cite

Seismic phase detection, identification and first-onset picking are basic but essential routines to analyse earthquake data. As both the number of seismic stations, globally and regionally, and the number of experiments greatly increase due to ever greater availability of instrumentation, automated data processing becomes more and more essential. E.g., for modern seismic experiments involving 100s to even 1,000s instruments, conventional human analyst-based identification and picking of seismic phases is becoming unfeasible, and the introduction of automatic algorithms mandatory. In this paper, we introduce DeepPhasePick, an automatic two-stage method that detects and picks P and S seismic phases from local earthquakes. The method is entirely based on highly optimized deep neural networks, consisting of a first stage that detects the phases using a convolutional neural network, and a second stage that uses two recurrent neural networks to pick both phases. Detection is performed on three-component seismograms. P-and S-picking is then conducted on the vertical and the two-horizontal components, respectively. Systematic hyperparameter optimization was applied to select the best model architectures and to define both the filter applied to preprocess the seismic data as well as the characteristics of the window sample used to feed the models. We trained DeepPhasePick using seismic records extracted from two sets of manually-picked event waveforms originating from northern Chile (∼39,000 records for detection and ∼36,000 records for picking). In dif-Page 1 of 60 Geophysical Journal International 2 H. Soto & B. Schurr ferent tectonic regimes, DeepPhasePick demonstrated the ability to both detect P and S phases from local earthquakes with high accuracy, as well as predict P-and S-phase time onsets with an analyst level of precision. DeepPhasePick additionally computes onset uncertainties based on the Monte Carlo Dropout technique as an approximation of Bayesian inference. This information can then further feed an associator algorithm in an earthquake location procedure.

show abstract

“…Most of such techniques are based on learning curve extrapolation [25] and surrogate models using RNN predictor [52] that aim at predicting and eliminating poor architectures before full training. Another idea to estimate performance and rank designed architectures is to use simplified (proxy) metrics for training such as data subsets (mini-batches) [29] and down-sampled data (like images with lower resolution) [53].…”

Section: Architecture Search Acceleratorsmentioning

confidence: 99%

Reinforcement learning for neural architecture search: A review

Jaafra

Laurent²,

Deruyver

et al. 2019

Image and Vision Computing

121

View full text Add to dashboard Cite

Deep Neural networks are efficient and flexible models that perform well for a variety of tasks such as image, speech recognition and natural language understanding. In particular, convolutional neural networks (CNN) generate a keen interest among researchers in computer vision and more specifically in classification tasks. CNN architecture and related hyperparameters are generally correlated to the nature of the processed task as the network extracts complex and relevant characteristics allowing the optimal convergence. Designing such architectures requires significant human expertise, substantial computation time and doesnt always lead to the optimal network. Model configuration topic has been extensively studied in machine learning without leading to a standard automatic method. This survey focuses on reviewing and discussing the current progress in automating CNN architecture search.

show abstract

Speeding up the Hyperparameter Optimization of Deep Convolutional Neural Networks

Cited by 102 publications

References 22 publications

Image Synthesis in Multi-Contrast MRI With Conditional Generative Adversarial Networks

Image Synthesis in Multi-Contrast MRI With Conditional Generative Adversarial Networks

DeepPhasePick: A method for Detecting and Picking Seismic Phases from Local Earthquakes based on highly optimized Convolutional and Recurrent Deep Neural Networks

Reinforcement learning for neural architecture search: A review

Contact Info

Product

Resources

About