HAL: Computer System for Scalable Deep Learning

Kindratenko, Volodymyr; Mu, Dawei; Yan, Zhan You; Maloney, J. D.; Hashemi, Seyed Mohammad; Rabe, Benjamin; Xu, Ke; Campbell, Roy H.; Peng, Jian; Gropp, William

doi:10.1145/3311790.3396649

Cited by 52 publications

(27 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…All experiments in this study were run on gpux1 on the HAL (Hardware-Accelerated Learning) cluster [54].…”

Section: Methodsmentioning

confidence: 99%

Classifying COVID-19 variants based on genetic sequences using deep learning models

Basu

Campbell

2021

Preprint

View full text Add to dashboard Cite

The COrona VIrus Disease (COVID-19) pandemic led to the occurrence of several variants with time. This has led to an increased importance of understanding sequence data related to COVID-19. In this chapter, we propose an alignment-free k-mer based LSTM (Long Short-Term Memory) deep learning model that can classify 20 different variants of COVID-19. We handle the class imbalance problem by sampling a fixed number of sequences for each class label. We handle the vanishing gradient problem in LSTMs arising from long sequences by dividing the sequence into fixed lengths and obtaining results on individual runs. Our results show that one- vs-all classifiers have test accuracies as high as 92.5% with tuned hyperparameters compared to the multi-class classifier model. Our experiments show higher overall accuracies for B.1.1.214, B.1.177.21, B.1.1.7, B.1.526, and P.1 on the one-vs-all classifiers, suggesting the presence of distinct mutations in these variants. Our results show that embedding vector size and batch sizes have insignificant improvement in accuracies, but changing from 2-mers to 3-mers mostly improves accuracies. We also studied individual runs which show that most accuracies improved after the 20th run, indicating that these sequence positions may have more contributions to distinguishing among different COVID-19 variants.

show abstract

“…All experiments in this study were run on gpux1 on the HAL (Hardware-Accelerated Learning) cluster [54].…”

Section: Methodsmentioning

confidence: 99%

Classifying COVID-19 variants based on genetic sequences using deep learning models

Basu

Campbell

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…This work was supported in part by NIH awards EB020604, EB023045, NS102213, EB028652 and the Washington University Imaging Sciences Pathway Trainee Fellowship (5T32EB01485505). This work utilized resources supported by the National Science Foundation's Major Research Instrumentation program, grant #1725729, as well as the University of Illinois at Urbana-Champaign [45].…”

Section: Acknowledgementsmentioning

confidence: 99%

A Method for Evaluating the Capacity of Generative Adversarial Networks to Reproduce High-order Spatial Context

Deshpande¹,

Anastasio²,

Brooks³

2021

Preprint

View full text Add to dashboard Cite

Generative adversarial networks (GANs) are generative models with the potential to revolutionize biomedical imaging. The overarching problem with GANs in clinical applications is a lack of adequate or automatic means of assessing the diagnostic quality of images generated by GANs. We demonstrate several statistical tests of images output by two popular GAN architectures. We designed several stochastic object models (SOMs) of distinct features that can be recovered after generation by a trained GAN. Several of these features are high-order, algorithmic pixel-arrangement rules which are not readily expressed in covariance matrices. We designed and validated statistical classifiers to detect the known arrangement rules, and tested the rates at which the GANs correctly reproduced the rules under a variety of training scenarios. We found that ensembles of generated images can appear accurate visually, and correspond to low Fr èchet Inception Distance, while not exhibiting the known spatial arrangements. Additionally, GANs trained on a spectrum of distinct spatial orders did not respect the order prevalence given in the training data. The main conclusion is that while low-order ensemble statistics are largely correct, there are numerous quantifiable errors per image that plausibly can affect subsequent use of the GAN-generated images.

show abstract

“…Their computational resources and support from their staff were essential to the successful completion of our project. In addition, this work utilized resources supported by the National Science Foundation's Major Research Instrumentation program, grant 1725729, as well as the University of Illinois at Urbana-Champaign [15]. We'd like to acknowledge the work and authors of Anatome, TorchMeta and higher [13,7,11] for making their code available and answering ours questions in their project's GitHub repository.…”

Section: Acknowledgments and Disclosure Of Fundingmentioning

confidence: 99%

Does MAML Only Work via Feature Re-use? A Data Centric Perspective

Miranda¹,

Wang²,

Koyejo³

2021

Preprint

View full text Add to dashboard Cite

Recent work has suggested that a good embedding is all we need to solve many few-shot learning benchmarks. Furthermore, other work has strongly suggested that Model Agnostic Meta-Learning (MAML) also works via this same method -by learning a good embedding. These observations highlight our lack of understanding of what meta-learning algorithms are doing and when they work. In this work, we provide empirical results that shed some light on how meta-learned MAML representations function. In particular, we identify three interesting properties: 1) In contrast to previous work, we show that it is possible to define a family of synthetic benchmarks that result in a low degree of feature re-use -suggesting that current few-shot learning benchmarks might not have the properties needed for the success of meta-learning algorithms; 2) meta-overfitting occurs when the number of classes (or concepts) are finite, and this issue disappears once the task has an unbounded number of concepts (e.g., online learning); 3) more adaptation at metatest time with MAML does not necessarily result in a significant representation change or even an improvement in meta-test performance -even when training on our proposed synthetic benchmarks. Finally, we suggest that to understand metalearning algorithms better, we must go beyond tracking only absolute performance and, in addition, formally quantify the degree of meta-learning and track both metrics together. Reporting results in future work this way will help us identify the sources of meta-overfitting more accurately and help us design more flexible metalearning algorithms that learn beyond fixed feature re-use. Finally, we conjecture the core challenge of re-thinking meta-learning is in the design of few-shot learning data sets and benchmarks -rather than in the algorithms, as suggested by previous work.

show abstract

HAL: Computer System for Scalable Deep Learning

Cited by 52 publications

References 3 publications

Classifying COVID-19 variants based on genetic sequences using deep learning models

Classifying COVID-19 variants based on genetic sequences using deep learning models

A Method for Evaluating the Capacity of Generative Adversarial Networks to Reproduce High-order Spatial Context

Does MAML Only Work via Feature Re-use? A Data Centric Perspective

Contact Info

Product

Resources

About