OCEAN: An on-chip incremental-learning enhanced processor with gated recurrent neural network accelerators

Chen, Chixiao; Ding, Hongwei; Peng, Huwan; Zhu, Haozhe; Ma, Rui; Zhang, Peiyong; Yan, Xiaolang; Wang, Yu; Wang, Mingyu; Min, Hao; Shi, Richard

doi:10.1109/esscirc.2017.8094575

Cited by 22 publications

(9 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While z t can be computed in parallel as it is independent of h t and r t . The same hardware can be shared for computing r t and z t to save hardware resources [97].…”

Section: ) Compute-specificmentioning

confidence: 99%

“…Papers that do not discuss any flexibility aspects are omitted from Table 10. In A4 [97], the architecture should be able to support various models, but the number of cells and layers the architecture can support are not mentioned in the paper. Hence, we cannot deduce how the implementation could support variations in the RNN model.…”

Section: B Flexibilitymentioning

confidence: 99%

“…Incremented online training was included in A4 [97] to support retraining pre-trained networks to enhance accuracy. Changes in hardware design were applied to support both training and inference without affecting the quality of inference.…”

Section: ) Online Training (Level 4)mentioning

confidence: 99%

See 2 more Smart Citations

Recurrent Neural Networks: An Embedded Computing Perspective

et al. 2020

View full text Add to dashboard Cite

Recurrent Neural Networks (RNNs) are a class of machine learning algorithms used for applications with time-series and sequential data. Recently, there has been a strong interest in executing RNNs on embedded devices. However, difficulties have arisen because RNN requires high computational capability and a large memory space. In this paper, we review existing implementations of RNN models on embedded platforms and discuss the methods adopted to overcome the limitations of embedded systems. We will define the objectives of mapping RNN algorithms on embedded platforms and the challenges facing their realization. Then, we explain the components of RNN models from an implementation perspective. We also discuss the optimizations applied to RNNs to run efficiently on embedded platforms. Finally, we compare the defined objectives with the implementations and highlight some open research questions and aspects currently not addressed for embedded RNNs. Overall, applying algorithmic optimizations to RNN models and decreasing the memory access overhead is vital to obtain high efficiency. To further increase the implementation efficiency, we point up the more promising optimizations that could be applied in future research. Additionally, this article observes that high performance has been targeted by many implementations, while flexibility has, as yet, been attempted less often. Thus, the article provides some guidelines for RNN hardware designers to support flexibility in a better manner. INDEX TERMS Compression, flexibility, efficiency, embedded computing, long short term memory (LSTM), quantization, recurrent neural networks (RNNs).

show abstract

“…While z t can be computed in parallel as it is independent of h t and r t . The same hardware can be shared for computing r t and z t to save hardware resources [97].…”

Section: ) Compute-specificmentioning

confidence: 99%

Section: B Flexibilitymentioning

confidence: 99%

See 1 more Smart Citation

Recurrent Neural Networks: An Embedded Computing Perspective

et al. 2020

View full text Add to dashboard Cite

show abstract

“…One of the most popular approaches to obtain more energy efficient inference for neural networks is through custom hardware accelerators, targeting field-programmable gate arrays (FPGAs) [15,19,39] or application-specific integrated circuit (ASICs) [3,6,28,40]. These are custom-built architectures that optimize the most energy-intensive operations involved in the inference process (typically multiply-and-accumulate loops).…”

Section: Custom Hardware Designsmentioning

confidence: 99%

Sequence-To-Sequence Neural Networks Inference on Embedded Processors Using Dynamic Beam Search

2020

View full text Add to dashboard Cite

Sequence-to-sequence deep neural networks have become the state of the art for a variety of machine learning applications, ranging from neural machine translation (NMT) to speech recognition. Many mobile and Internet of Things (IoT) applications would benefit from the ability of performing sequence-to-sequence inference directly in embedded devices, thereby reducing the amount of raw data transmitted to the cloud, and obtaining benefits in terms of response latency, energy consumption and security. However, due to the high computational complexity of these models, specific optimization techniques are needed to achieve acceptable performance and energy consumption on single-core embedded processors. In this paper, we present a new optimization technique called dynamic beam search, in which the inference complexity is tuned to the difficulty of the processed input sequence at runtime. Results based on measurements on a real embedded device, and on three state-of-the-art deep learning models, show that our method is able to reduce the inference time and energy by up to 25% without loss of accuracy.

show abstract

“…To lower this burden, hardware accelerators have been developed targeting Keyword Spotting. [2] shows an ASIC with 8 execution engines for accelerating RNNs, showing its application for realtime KWS, consuming 6.6 mW at 20 MHz/0.8 V. A compact memory storage is presented in [3] through a programmable processor with 270 kB on-chip weight which executes realtime KWS with 300 μW power consumption. In [4] a SIMD processor evaluates DNNs for automatic speech recognition tasks including a small vocabulary recognizer which achieves 172 μW for real time power consumption.…”

Section: Introductionmentioning

confidence: 99%

Laika: A 5uW Programmable LSTM Accelerator for Always-on Keyword Spotting in 65nm CMOS

Giraldo

Verhelst

2018

ESSCIRC 2018 - IEEE 44th European Solid State Circuits Conference (ESSCIRC)

View full text Add to dashboard Cite

The ubiquitous importance of speech recognition for diverse applications in mobile devices, necessitates its low power embedded execution. Often, a Keyword Spotting System (KWS) is used to detect specific wake-up words spoken by a user, as a simple user interface, or front-end layer to a larger speech recognition system. Yet, such KWS must be always active, hence imposing strict power and latency constraints. While deep learning algorithms like Long Short-Term Memory (LSTM) demonstrated excellent KWS accuracies, current implementations fail to fit in the tight embedded memory and power budgets.This paper presents Laika: the implementation of a KWS system using an LSTM accelerator designed in 65nm CMOS. For this application, an LSTM model is trained through a speech database and deployed on our custom, yet highly programmable LSTM accelerator. Approximate computing techniques further reduce power consumption, while maintaining high accuracy and reliability. Experimental results demonstrate a power consumption of less than 5μW for real-time KWS applications.

show abstract

OCEAN: An on-chip incremental-learning enhanced processor with gated recurrent neural network accelerators

Cited by 22 publications

References 7 publications

Recurrent Neural Networks: An Embedded Computing Perspective

Recurrent Neural Networks: An Embedded Computing Perspective

Sequence-To-Sequence Neural Networks Inference on Embedded Processors Using Dynamic Beam Search

Laika: A 5uW Programmable LSTM Accelerator for Always-on Keyword Spotting in 65nm CMOS

Contact Info

Product

Resources

About