An Empirical Analysis of Generative Adversarial Network Training Times with Varying Batch Sizes

Ghosh, Bhaskar; Dutta, Indira Kalyan; Carlson, Albert; Totaro, Michael W.; Bayoumi, Magdy

doi:10.1109/uemcon51285.2020.9298092

Cited by 8 publications

(8 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, we had tested ADAN on a very limited dataset. Because GANs are notoriously sensitive to hyperparameter settings (18, 22, 27), it was unclear how robust ADAN would be in practice. Another promising method, PAF, had been tested primarily in terms of two monkeys’ online iBCI performance (9).…”

Section: Discussionmentioning

confidence: 99%

“…Correct optimization of GANs is also directly linked to proper tuning of the dynamics of learning during training (27, 41), which we investigated here in depth. Given the many GAN variants, there are still no comprehensive guidelines for a particular architecture (22). Consistent with this, we found that ADAN and Cycle-GAN differ substantially in their sensitivity to learning rate and batch size hyperparameters.…”

Section: Discussionmentioning

confidence: 99%

“…High sensitivity of model performance to hyperparameter values would pose a potential barrier to the adoption of either ADAN or Cycle-GAN as a tool for cross-day alignment. As in (22), we assessed sensitivity to hyperparameters by testing the impact of batch size and learning rates on alignment performance. Because these hyperparameter sweeps are very computationally expensive, we evaluated them using only the single dataset with the greatest span of time.…”

Section: Cycle-gan Is Robust To Hyperparameter Settingsmentioning

confidence: 99%

See 2 more Smart Citations

Using adversarial networks to extend brain computer interface decoding accuracy over time

Ma¹,

Rizzoglio²,

Perreault³

et al. 2022

Preprint

View full text Add to dashboard Cite

Existing intracortical brain computer interfaces (iBCIs) transform neural activity into control signals capable of restoring movement to persons with paralysis. However, the accuracy of the "decoder" at the heart of the iBCI typically degrades over time due to turnover of recorded neurons. To compensate, decoders can be recalibrated, but this requires the user to spend extra time and effort to provide the necessary data, then learn the new dynamics. As the recorded neurons change, one can think of the underlying movement intent signal being expressed in changing coordinates. If a mapping can be computed between the different coordinate systems, it may be possible to stabilize the original decoder's mapping from brain to behavior without recalibration. We previously proposed a method based on Generalized Adversarial Networks (GANs), called "Adversarial Domain Adaptation Network" (ADAN), which aligns the distributions of latent signals within underlying low-dimensional neural manifolds. However, ADAN was tested on only a very limited dataset. Here we propose a method based on Cycle-Consistent Adversarial Networks (Cycle-GAN), which aligns the distributions of the full-dimensional neural recordings. We tested both Cycle-GAN and ADAN on data from multiple monkeys and behaviors and compared them to a linear method based on Procrustes Alignment of axes provided by Factor Analysis (PAF). Both GAN-based methods outperformed PAF. Cycle-GAN and ADAN (like PAF) are unsupervised and require little data, making them practical in real life. Overall, Cycle-GAN had the best performance and was easier to train and more robust than ADAN, making it ideal for stabilizing iBCI systems over time.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Cycle-gan Is Robust To Hyperparameter Settingsmentioning

confidence: 99%

See 1 more Smart Citation

Using adversarial networks to extend brain computer interface decoding accuracy over time

Ma¹,

Rizzoglio²,

Perreault³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…For readers familiar with ordinal logistic regression, Datawig uses the same encoding to capture the Deep Learning Imputation for Likert-Type Items FIGURE 6. Stochastic gradient descent (Ghosh et al, 2020). Note.…”

Section: Hyperparametersmentioning

confidence: 99%

Deep Learning Imputation for Asymmetric and Incomplete Likert-Type Items

Collier¹,

Kong²,

Soyoye³

et al. 2023

Journal of Educational and Behavioral Statistics

View full text Add to dashboard Cite

Asymmetric Likert-type items in research studies can present several challenges in data analysis, particularly concerning missing data. These items are often characterized by a skewed scaling, where either there is no neutral response option or an unequal number of possible positive and negative responses. The use of conventional techniques, such as discriminant analysis or logistic regression imputation, for handling missing data in asymmetric items may result in significant bias. It is also recommended to exercise caution when employing alternative strategies, such as listwise deletion or mean imputation, because these methods rely on assumptions that are often unrealistic in surveys and rating scales. This article explores the potential of implementing a deep learning-based imputation method. Additionally, we provide access to deep learning-based imputation to a broader group of researchers without requiring advanced machine learning training. We apply the methodology to the Wilmington Street Participatory Action Research Health Project.

show abstract

“…Although a high-performance GP-GPU often allows for higher batch Size values, other factors must be considered when dealing with a GAN. As pointed out by [21] and [20], a value of batch Size that is too large for a GAN could significantly increase training time and affect the overall performance of the model, potentially causing a decrease of its performance. Therefore, small values are recommended to be used so as to obtain better performance of the network.…”

Section: Training Settingsmentioning

confidence: 99%

Image Super Resolution Using Generative Adversarial Networks and non-Paired Strategy

Moreira¹,

Romero²,

Ribeiro³

2021

Anais Do 15. Congresso Brasileiro De Inteligência Computacional

View full text Add to dashboard Cite

The quality of images obtained from video surveillance systems is a decisive aspect when performing investigations at the Forensic Science. Features such as scars, tattoos, and skin marks are great examples of details that allow to consolidate an investigation at certain scenarios in which there is the necessity to identify individuals captured in a video or image footage. However, the low quality of images could affect the results of the investigations. In this sense, this work proposes the study of a computational model to address the problem of increasing the resolution of Low-Resolution (LR) images, also known as the problem of super-resolution of images. The main idea is to train a Generative Adversarial Network (GAN) so that it can be able to enhance low-quality images. The hypothesis is that a variant model of a GAN, named Super-Resolution Generative Adversarial Network (SRGAN), is capable to produce High-Resolution (HR) images from LR ones. The proposed methodology is based on experimental research with the aid of the hypothetical deductive method, where two well-recognised state of art methods were used, which proposes the use of convolutional neural networks and deep learning. For the model validation, were conducted four different experiments: two to avail the capacity of the GAN to produce images with enhanced resolution and two other experiments to evaluate the quality of the results produced by the SRGAN. The quantitative results of our experiments are promising, with performances that are similar to those obtained by state-of-the-art approaches. Moreover, the qualitative results based on performing a visual analysis of the images produced by our approach suggest a interesting performance in terms of visual quality.

show abstract

An Empirical Analysis of Generative Adversarial Network Training Times with Varying Batch Sizes

Cited by 8 publications

References 7 publications

Using adversarial networks to extend brain computer interface decoding accuracy over time

Using adversarial networks to extend brain computer interface decoding accuracy over time

Deep Learning Imputation for Asymmetric and Incomplete Likert-Type Items

Image Super Resolution Using Generative Adversarial Networks and non-Paired Strategy

Contact Info

Product

Resources

About