Data Parallel Large Sparse Deep Neural Network on GPU

Sattar, Naw Safrin; Anfuzzaman, Shaikh

doi:10.1109/ipdpsw50202.2020.00170

Cited by 6 publications

(3 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Time series forecasting is the process of using a model to generate predictions (forecasts) for future events based on known past events [22,37]. There are several machine learning methods:s regression, classification, clustering [38], dimensionality reduction, ensemble methods, neural nets and deep learning [39], transfer learning, reinforcement learning, Natural Language Processing (NLP),word embeddings, etc. Regression is one of the predictive modeling techniques which analyzes the correlations between a target and independent variables.…”

Section: Time Series Forecastingmentioning

confidence: 99%

COVID-19 Vaccination Awareness and Aftermath: Public Sentiment Analysis on Twitter Data and Vaccinated Population Prediction in the USA

Sattar

Arifuzzaman

2021

Applied Sciences

Self Cite

View full text Add to dashboard Cite

Social media, such as Twitter, is a source of exchanging information and opinion on global issues such as COVID-19 pandemic. In this study, we work with a database of around 1.2 million tweets collected across five weeks of April–May 2021 to draw conclusions about public sentiments towards the vaccination outlook when vaccinations become widely available to the population during the COVID-19 pandemic. We deploy natural language processing and sentiment analysis techniques to reveal insights about COVID-19 vaccination awareness among the public. Our results show that people have positive sentiments towards taking COVID-19 vaccines instead of some adverse effects of some of the vaccines. We also analyze people’s attitude towards the safety measures of COVID-19 after receiving the vaccines. Again, the positive sentiment is higher than that of negative in terms of maintaining safety measures against COVID-19 among the vaccinated population. We also project that around 62.44% and 48% of the US population will get at least one dose of vaccine and be fully vaccinated, respectively, by the end of July 2021 according to our forecast model. This study will help to understand public reaction and aid the policymakers to project the vaccination campaign as well as health and safety measures in the ongoing global health crisis.

show abstract

Section: Time Series Forecastingmentioning

confidence: 99%

COVID-19 Vaccination Awareness and Aftermath: Public Sentiment Analysis on Twitter Data and Vaccinated Population Prediction in the USA

Sattar

Arifuzzaman

2021

Applied Sciences

Self Cite

View full text Add to dashboard Cite

show abstract

“…The neural network parallelization process can occur during feed forward and backpropagation. It is because each node in a layer does not need information from other nodes in the same layer, so the process can run in parallel [34]. The first model only used the CPU in the modeling process because the processes were sequential.…”

Section: Build Prediction Modelmentioning

confidence: 99%

Deep learning optimization for drug-target interaction prediction in COVID-19 using graphic processing unit

Darmawan¹,

Kusuma²,

Rahmawan³

2023

IJECE

View full text Add to dashboard Cite

<span lang="EN-US">The exponentially increasing bioinformatics data raised a new problem: the computation time length. The amount of data that needs to be processed is not matched by an increase in hardware performance, so it burdens researchers on computation time, especially on drug-target interaction prediction, where the computational complexity is exponential. One of the focuses of high-performance computing research is the utilization of the graphics processing unit (GPU) to perform multiple computations in parallel. This study aims to see how well the GPU performs when used for deep learning problems to predict drug-target interactions. This study used the gold-standard data in drug-target interaction (DTI) and the coronavirus disease (COVID-19) dataset. The stages of this research are data acquisition, data preprocessing, model building, hyperparameter tuning, performance evaluation and COVID-19 dataset testing. The results of this study indicate that the use of GPU in deep learning models can speed up the training process by 100 times. In addition, the hyperparameter tuning process is also greatly helped by the presence of the GPU because it can make the process up to 55 times faster. When tested using the COVID-19 dataset, the model showed good performance with 76% accuracy, 74% F-measure and a speed-up value of 179.</span>

show abstract

“…In this regard, one significant advantage of sparse models is that the sparse gradient communication is automatically at hand. Related work on parallelisation for sparse DNN is presented in (Sattar & Anfuzzaman (2020)) as a solution to the Sparse DNN Challenge posed by MIT/IEEE/Amazon. However, their work is focused on sparse neural networks created using RadiX-Net (Kepner & Robinett (2019)) which do not evolve the topology over time.…”

Section: Parallel Training Of Sparse Networkmentioning

confidence: 99%

Truly Sparse Neural Networks at Scale

Curci

Mocanu

Pechenizkiy

2021

Preprint

View full text Add to dashboard Cite

Recently, sparse training methods have started to be established as a de facto approach for training and inference efficiency in artificial neural networks. Yet, this efficiency is just in theory. In practice, everyone uses a binary mask to simulate sparsity since the typical deep learning software and hardware are optimized for dense matrix operations. In this paper, we take an orthogonal approach, and we show that we can train truly sparse neural networks to harvest their full potential. To achieve this goal, we introduce three novel contributions, specially designed for sparse neural networks: (1) a parallel training algorithm and its corresponding sparse implementation from scratch, (2) an activation function with non-trainable parameters to favour the gradient flow, and (3) a hidden neurons importance metric to eliminate redundancies. All in one, we are able to break the record and to train the largest neural network ever trained in terms of representational power -- reaching the bat brain size. The results show that our approach has state-of-the-art performance while opening the path for an environmentally friendly artificial intelligence era.

show abstract

Data Parallel Large Sparse Deep Neural Network on GPU

Cited by 6 publications

References 18 publications

COVID-19 Vaccination Awareness and Aftermath: Public Sentiment Analysis on Twitter Data and Vaccinated Population Prediction in the USA

COVID-19 Vaccination Awareness and Aftermath: Public Sentiment Analysis on Twitter Data and Vaccinated Population Prediction in the USA

Deep learning optimization for drug-target interaction prediction in COVID-19 using graphic processing unit

Truly Sparse Neural Networks at Scale

Contact Info

Product

Resources

About