Scaling Analysis of Specialized Tensor Processing Architectures for Deep Learning Models

Gordienko, Yuri; Kochura, Yuriy; Taran, Vlad; Gordienko, Nikita; Rokovyi, Alexandr; Alienin, Oleg; Stirenko, Sergii

doi:10.1007/978-3-030-31756-0_3

Cited by 23 publications

(3 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It should be noted that these results were obtained without detriment to the accuracy and loss for the relatively simple DNN and small images (28x28). The current investigations of network size impact and image size impact are under work now and their results will be published elsewhere [15]. In addition to Google TPU architecture, the specific tensor processing hardware tools are available in the other modern GPUcards like Tesla V100 and Titan V by NVIDIA based on the Volta microarchitecture with specialized Tensor Cores Units (640 TCU) and their influence on training and prediction speedup are under investigation and will be reported elsewhere [15].…”

Section: Discussionmentioning

confidence: 99%

“…As far as the model size limits the available memory space for the batch of images other techniques could be useful for squeezing the model size, like quantization [15][16] and pruning [17][18]. These results can be used for selection of the optimal parameters for applications where a large batch of data needs to be processed, for example, for monitoring real-time road condition in advanced driver assistance systems (ADAS), where such specialized architectures like TPU, TCU, and FPGA-based solutions can be deployed [19].…”

Section: Discussionmentioning

confidence: 99%

“…It is explained by the inability to use batch size>8192 for the current datasets with images of sizes 28x28. For the bigger images the maximally possible batch size will be lower and the speedup will be saturated earlier, but it will be investigated elsewhere [15].…”

Section: Tpumentioning

confidence: 99%

See 2 more Smart Citations

Batch Size Influence on Performance of Graphic and Tensor Processing Units During Training and Inference Phases

Kochura

Gordienko

Taran

et al. 2019

Advances in Computer Science for Engineering and Education II

Self Cite

View full text Add to dashboard Cite

The impact of the maximally possible batch size (for the better runtime) on performance of graphic processing units (GPU) and tensor processing units (TPU) during training and inference phases is investigated. The numerous runs of the selected deep neural network (DNN) were performed on the standard MNIST and Fashion-MNIST datasets. The significant speedup was obtained even for extremely low-scale usage of Google TPUv2 units (8 cores only) in comparison to the quite powerful GPU NVIDIA Tesla K80 card with the speedup up to 10x for training stage (without taking into account the overheads) and speedup up to 2x for prediction stage (with and without taking into account overheads). The precise speedup values depend on the utilization level of TPUv2 units and increase with the increase of the data volume under processing, but for the datasets used in this work (MNIST and Fashion-MNIST with images of sizes 28x28) the speedup was observed for batch sizes >512 images for training phase and >40 000 images for prediction phase. It should be noted that these results were obtained without detriment to the prediction accuracy and loss that were equal for both GPU and TPU runs up to the 3rd significant digit for MNIST dataset, and up to the 2nd significant digit for Fashion-MNIST dataset.

show abstract

Section: Discussionmentioning

confidence: 99%