GEO: Generation and Execution Optimized Stochastic Computing Accelerator for Neural Networks

Li, Tianmu; Romaszkan, Wojciech; Pamarti, Sudhakar; Gupta, Puneet

doi:10.23919/date51398.2021.9473911

“…[22] Non-standard 6 7 Ref. [24] Non-standard 4 6-7 Ref. [33] Non-standard N/A 8 Approach Network Depth Precision (bits) Ref.…”

Section: Inference Examplesmentioning

confidence: 99%

“…IV (Table 2) are compared with previous work in Tables 3 and 4, for CIFAR10 and ImageNet classification, respectively. The CIFAR10 classification results reported on stochastic computing hardware [22], [24], [33] were obtained on nonstandard and/or small (4−6 layer) networks and required 6−8 bits of precision. The results from Sec.…”

Section: Comparison With Previous Workmentioning

confidence: 99%

“…Another challenge is the area overhead associated with the use of RNGs typically implemented with linear feedback shift registers (LFSRs), which cannot be shared as the pseudorandom sequences need to be uncorrelated. Due to these challenges, recent studies of stochastic computing have been largely focused on convolutional neural networks where application of stochastic computing is expected to have the highest impact, and primarily on small networks, and datasets such as MNIST [9]− [24], and to some extent on somewhat larger datasets such as CIFAR10 [22]− [24]. Application to larger networks, and datasets such as ImageNet has generally been considered impractical [20], [24], [34].…”

Section: Introductionmentioning

confidence: 99%

“…Due to these challenges, recent studies of stochastic computing have been largely focused on convolutional neural networks where application of stochastic computing is expected to have the highest impact, and primarily on small networks, and datasets such as MNIST [9]− [24], and to some extent on somewhat larger datasets such as CIFAR10 [22]− [24]. Application to larger networks, and datasets such as ImageNet has generally been considered impractical [20], [24], [34]. This can be seen from the inference examples given in Fig.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Deterministic Approach to Approximate Computing on Stochastic Computing Hardware, with Reduced Sequence Lengths for Convolutional Neural Networks

Hekmatshoar¹

2023

Preprint

0

View full text Add to dashboard Cite

<p>In this work, a deterministic sequence suitable for approximate computing on stochastic computing hardware is proposed and its effectiveness in achieving high accuracies with relatively short sequence lengths is studied for convolutional neural networks. It is shown that in the range of interest for neural network computations, multiplication errors can be lower than quantization errors with this approach. The sequence lengths required for achieving accuracies within ~0.5% of the floating-point baseline are of the order of 16 and 32 for CIFAR10 classification with VGG16 and ResNet20 networks, respectively, when all convolutions and matrix multiplications are performed using the proposed sequence. For ImageNet classification, the sequence lengths required for accuracies within ~1% of the floating-point baseline are of the order of 32 for MobileNetV1 and ResNet50 networks. This work suggests that stochastic computing hardware and approaches may be feasible for approximate neural network computations with higher accuracies, lower latencies and/or larger networks than previously reported. </p>

show abstract

“…Another challenge is the area overhead associated with the use of RNGs typically implemented with linear feedback shift registers (LFSRs), which cannot be shared as the pseudorandom sequences need to be uncorrelated. Due to these challenges, recent studies of stochastic computing have been largely focused on convolutional neural networks where application of stochastic computing is expected to have the highest impact, and primarily on small networks, and datasets such as MNIST [9]− [24], and to some extent on somewhat larger datasets such as CIFAR10 [22]− [24]. Application to larger networks, and datasets such as ImageNet has generally been considered impractical [20], [24], [34].…”

Section: Introductionmentioning

confidence: 99%

A Deterministic Approach to Approximate Computing on Stochastic Computing Hardware, with Reduced Sequence Lengths for Convolutional Neural Networks

Hekmatshoar¹

2023

Preprint

0

View full text Add to dashboard Cite

<p>In this work, a deterministic sequence suitable for approximate computing on stochastic computing hardware is proposed and its effectiveness in achieving high accuracies with relatively short sequence lengths is studied for convolutional neural networks. It is shown that in the range of interest for neural network computations, multiplication errors can be lower than quantization errors with this approach. The sequence lengths required for achieving accuracies within ~0.5% of the floating-point baseline are of the order of 16 and 32 for CIFAR10 classification with VGG16 and ResNet20 networks, respectively, when all convolutions and matrix multiplications are performed using the proposed sequence. For ImageNet classification, the sequence lengths required for accuracies within ~1% of the floating-point baseline are of the order of 32 for MobileNetV1 and ResNet50 networks. This work suggests that stochastic computing hardware and approaches may be feasible for approximate neural network computations with higher accuracies, lower latencies and/or larger networks than previously reported. </p>

show abstract

Reconfigurable spatial-parallel stochastic computing for accelerating sparse convolutional neural networks

Xia

¹

,

Wan

²

,

Chen

³

et al. 2023

Sci. China Inf. Sci.

View full text Add to dashboard Cite

GEO: Generation and Execution Optimized Stochastic Computing Accelerator for Neural Networks

Cited by 8 publications

References 20 publications

A Deterministic Approach to Approximate Computing on Stochastic Computing Hardware, with Reduced Sequence Lengths for Convolutional Neural Networks

A Deterministic Approach to Approximate Computing on Stochastic Computing Hardware, with Reduced Sequence Lengths for Convolutional Neural Networks

A Deterministic Approach to Approximate Computing on Stochastic Computing Hardware, with Reduced Sequence Lengths for Convolutional Neural Networks

Reconfigurable spatial-parallel stochastic computing for accelerating sparse convolutional neural networks

Contact Info

Product

Resources

About