η-LSTM: Co-Designing Highly-Efficient Large LSTM Training via Exploiting Memory-Saving and Architectural Design Opportunities

Zhang, Xingyao; Xia, Haojun; Zhuang, Donglin; Sun, Hao; Fu, Xin; Taylor, Michael; Song, Shuaiwen Leon

doi:10.1109/isca52012.2021.00051

Cited by 5 publications

(2 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This section presents the performance results of the neural networks integrated into the three aforementioned scenarios. Four neural networks including LSTM 31 , GRU 32 , CNN 33 , and CNN-LSTM 34 are utilized in this study. Figure 12 illustrates the experimental settings of the aforementioned neural networks.…”

Section: Model Predictionsmentioning

confidence: 99%

Prediction of $$\beta $$-Thalassemia carriers using complete blood count features

Rustam

Ashraf

Jabbar

et al. 2022

Sci Rep

View full text Add to dashboard Cite

Abstract$$\beta $$ β -Thalassemia is one of the dangerous causes of the high mortality rate in the Mediterranean countries. Substantial resources are required to save a $$\beta $$ β -Thalassemia carriers’ life and early detection of thalassemia patients can help appropriate treatment to increase the carrier’s life expectancy. Being a genetic disease, it can not be prevented however the analysis of several indicators in parents’ blood can be used to detect disorders causing Thalassemia. Laboratory tests for Thalassemia are time-consuming and expensive like high-performance liquid chromatography, Complete Blood Count (CBC) with peripheral smear, genetic test, etc. Red blood indices from CBC can be used with machine learning models for the same task. Despite the available approaches for Thalassemia carriers from CBC data, gaps exist between the desired and achieved accuracy. Moreover, the data imbalance problem is studied well which makes the models less generalizable. This study proposes a highly accurate approach for $$\beta $$ β -Thalassemia detection using red blood indices from CBC augmented by supervised machine learning. In view of the fact that all the features do not carry predictive information regarding the target variable, this study employs a unified framework of two features selection techniques including Principal Component Analysis (PCA) and Singular Vector Decomposition (SVD). The data imbalance between $$\beta $$ β -Thalassemia carrier and non-carriers is handled by Synthetic Minority Oversampling Technique (SMOTE) and Adaptive Synthetic (ADASYN). Extensive experiments are performed using many state-of-the-art machine learning models and deep learning models. Experimental results indicate the superiority of the proposed approach over existing approaches with an accuracy score of 0.96.

show abstract

Section: Model Predictionsmentioning

confidence: 99%

Prediction of $$\beta $$-Thalassemia carriers using complete blood count features

Rustam

Ashraf

Jabbar

et al. 2022

Sci Rep

View full text Add to dashboard Cite

show abstract

“…Concerning GPU implementations, there is fewer related work as they usually pose a smaller research problem as FPGA implementations. However, there are quiet a few implementations [20,21] which focus on LSTM training on GPU platforms in order to reduce energy footprint and accelerate the training algorithm.…”

Section: Related Workmentioning

confidence: 99%

LSTM Acceleration with FPGA and GPU Devices for Edge Computing Applications in B5G MEC

Danopoulos

Stamoulias²,

Lentaris³

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

The advent of AI/ML in B5G and Multi-Access Edge Computing will rely on the acceleration of neural networks. The current work focuses on the acceleration of Long Short-Term Memory (LSTM) kernels playing a key role in numerous applications. We assume various LSTM sizes while targeting FPGA and GPU hardware for both embedded and server MEC purposes. Systematically, we perform a design space exploration to determine the most efficient acceleration approach and most suitable configuration for each device. We use High-Level-Synthesis to implement our proposed circuit architectures on Xilinx FPGAs, while we use high level tools for NVIDIA GPUs such as PyTorch's JIT compiler or ONNX runtime. Our exploration shows that the full parallelization of an LSTM array multiplication quickly overutilizes the FPGA, while on GPUs LSTM models can be deployed more easily. Instead, the best approach for FPGAs is to find a balance between parallelizing LSTM gates and vector multiplications. Our comparative study shows that FPGAs prevail in light LSTM models, whereas GPUs prevail in larger model topologies. Moreover, we show that far-and near-edge FPGAs achieve similar latency, however, near-edge GPUs can achieve one order of magnitude faster execution than far-edge GPUs. The best results range in 0.3-5msec latency per execution with acceleration factors in 12 × −174×.

show abstract

Marvel: Towards Efficient Federated Learning on IoT Devices

Liu,

2024

Computer Networks

View full text Add to dashboard Cite

η-LSTM: Co-Designing Highly-Efficient Large LSTM Training via Exploiting Memory-Saving and Architectural Design Opportunities

Cited by 5 publications

References 34 publications

Prediction of $$\beta $$-Thalassemia carriers using complete blood count features

Prediction of $$\beta $$-Thalassemia carriers using complete blood count features

LSTM Acceleration with FPGA and GPU Devices for Edge Computing Applications in B5G MEC

Marvel: Towards Efficient Federated Learning on IoT Devices

Contact Info

Product

Resources

About