Tiny and error-tolerant ConvLSTM for event-based vision sensor with optimized event representation and ReRAM computation-in-memory

Abstract

This paper proposes tiny and error tolerant Convolutional LSTM (ConvLSTM) for event-based vision sensor (EVS) data processing with Computation-in-Memory (CiM) that utilizes ReRAM. To handle the event data by ConvLSTM, the type and parameter of event representation are optimized to pre-process event data. To realize tiny and error tolerant ConvLSTM CiM, reduction of the bit precision and error tolerance in each ConvLSTM layer and weight are considered. On the other hand, reducing the bit precision degrades inference accuracy and error tolerance. ReRAM has non-idealities such as conductance variation and worn out by Set/Reset cycles that induces bit inversion. Thus, bit error rate at each Set/Reset cycles are investigated, and the bit precision of the weight is optimized to accept bit error rate of 0.1% in simulation. The proposed tiny ConvLSTM can reduce the size of memory by 91% compared with non-optimized ConvLSTM of that weight is represented in 32-bit precision.