CoIn: Accelerated CNN Co-Inference through data partitioning on heterogeneous devices

Vanishree, K; George, Adrian V.; Gunisetty, Srivatsav; Srinivasan, S.; R, Shravan Kashyap; Purnaprajna, Madhura

doi:10.1109/icaccs48705.2020.9074444

Cited by 5 publications

(3 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In an LSTM RNN, the recurrent cell is modified into the reset gate for Equation (3), forget gate for Equation (4), candidate memory cell for Equation (6), and output gate Equation (7). The internal diagram of an LSTM cell is shown in Figure 2.…”

Section: Lstm Rnnmentioning

confidence: 99%

“…Sequence modeling is an important subclass of machine learning problem. Sequence data involve a notion of time, and the learning of models such as RNNs that incorporate this aspect are far more complex than feed-forward, parallelizable, and spatial counterparts like Convolutional Neural Networks (CNNs) [5][6][7]. RNNs can process input data one at a time and remember information through their structure and hidden activations [8].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Systematic Evaluation of Recurrent Neural Network Models for Edge Intelligence and Human Activity Recognition Applications

Lalapura,

Bhimavarapu,

Amudha

et al. 2024

Algorithms

View full text Add to dashboard Cite

The Recurrent Neural Networks (RNNs) are an essential class of supervised learning algorithms. Complex tasks like speech recognition, machine translation, sentiment classification, weather prediction, etc., are now performed by well-trained RNNs. Local or cloud-based GPU machines are used to train them. However, inference is now shifting to miniature, mobile, IoT devices and even micro-controllers. Due to their colossal memory and computing requirements, mapping RNNs directly onto resource-constrained platforms is arcane and challenging. The efficacy of edge-intelligent RNNs (EI-RNNs) must satisfy both performance and memory-fitting requirements at the same time without compromising one for the other. This study’s aim was to provide an empirical evaluation and optimization of historic as well as recent RNN architectures for high-performance and low-memory footprint goals. We focused on Human Activity Recognition (HAR) tasks based on wearable sensor data for embedded healthcare applications. We evaluated and optimized six different recurrent units, namely Vanilla RNNs, Long Short-Term Memory (LSTM) units, Gated Recurrent Units (GRUs), Fast Gated Recurrent Neural Networks (FGRNNs), Fast Recurrent Neural Networks (FRNNs), and Unitary Gated Recurrent Neural Networks (UGRNNs) on eight publicly available time-series HAR datasets. We used the hold-out and cross-validation protocols for training the RNNs. We used low-rank parameterization, iterative hard thresholding, and spare retraining compression for RNNs. We found that efficient training (i.e., dataset handling and preprocessing procedures, hyperparameter tuning, and so on, and suitable compression methods (like low-rank parameterization and iterative pruning) are critical in optimizing RNNs for performance and memory efficiency. We implemented the inference of the optimized models on Raspberry Pi.

show abstract

Section: Lstm Rnnmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A Systematic Evaluation of Recurrent Neural Network Models for Edge Intelligence and Human Activity Recognition Applications

Lalapura,

Bhimavarapu,

Amudha

et al. 2024

Algorithms

View full text Add to dashboard Cite

show abstract

“…This solution is tailored to very constrained resources where communication is a dominating bottleneck. In [4], the previous use case is extended to heterogeneous platforms with different devices including CPUs, FPGAs and GPUs. Therefore, communication channels with several latencies and throughputs are considered in the optimization problem.…”

Section: Related Workmentioning

confidence: 99%

Automatic CNN Model Partitioning for GPU/FPGA-based Embedded Heterogeneous Accelerators using Geometric Programming

Carballo-Hernández,

Pelcat,

Berry

2023

J Sign Process Syst

View full text Add to dashboard Cite

Graphics Processing Unit (GPU), dedicated Application Specific Integrated Circuit (ASIC) and Field Programmable Gate Array (FPGA) accelerators are currently platforms of choice for porting Convolutional Neural Networks (CNNs). In this work, an automated Central Processing Unit (CPU)-GPU-FPGA partitioning selection is proposed for a given CNN layer. It is shown that using a Generalized Geometric Programming (GGP) optimization problem formulation, the CPU-GPU-FPGA partitioning problem can be modeled by considering a set of system performance metrics and constraints. Each metric is expressed in a posynomial form depending on CNN hyperparameters and architecture resource models. As for the partitioning method, the state-of-the-art techniques covered are: tiling, grouped convolution and fused-layer. The proposed analytical formalization is then employed to derive a set of objective functions and constraints as a GGP problem. It is demonstrated that it is possible to relax some problem constraints by including a penalization term, and reduce the problem to multiple simpler Geometric Programming (GP) sub-problems. Experimental results targeting an embedded FPGA-GPU platform with CNN layer configurations from state-of-the-art CNN models (AlexNet, VGG16 and ResNet18) show that the simplified problem is solvable in polynomial time with a speed-up gain and energy reduction of around 20% and 15%, respectively, when compared against an arbitrary balanced partitioning. If the models for objective and constraints functions preserve the posynomial form and log-log convexity, it is demonstrated that GGP is an efficient optimization solution to the Design Space Exploration (DSE) problem.

show abstract

References

2023

Accelerators for Convolutional Neural Networks

View full text Add to dashboard Cite

CoIn: Accelerated CNN Co-Inference through data partitioning on heterogeneous devices

Cited by 5 publications

References 11 publications

A Systematic Evaluation of Recurrent Neural Network Models for Edge Intelligence and Human Activity Recognition Applications

A Systematic Evaluation of Recurrent Neural Network Models for Edge Intelligence and Human Activity Recognition Applications

Automatic CNN Model Partitioning for GPU/FPGA-based Embedded Heterogeneous Accelerators using Geometric Programming

References

Contact Info

Product

Resources

About