Are 2D-LSTM really dead for offline text recognition?

Abstract

There is a recent trend in handwritten text recognition with deep neural networks to replace 2D recurrent layers with 1D, and in some cases even completely remove the recurrent layers, relying on simple feed-forward convolutional only architectures. The most used type of recurrent layer is the Long-Short Term Memory (LSTM). The motivations to do so are many:there are few open-source implementations of 2D-LSTM, even fewer supporting GPU implementations (currently cuDNN only implements 1D-LSTM); 2D recurrences reduce the amount of computations that can be parallelized, and thus possibly increase the training/inference time; recurrences create global dependencies with respect to the input, and sometimes this may not be desirable.Many recent competitions were won by systems that employed networks that use 2D-LSTM layers. Most previous work that compared 1D or pure feed-forward architectures to 2D recurrent models have done so on simple datasets or did not fully optimize the "baseline" 2D model compared to the challenger model, which was dully optimized.In this work, we aim at a fair comparison between 2D and competing models and also extensively evaluate them on more complex datasets that are more representative of challenging "real-world" data, compared to "academic" datasets that are more restricted in their complexity. We aim at determining when and why the 1D and 2D recurrent models have different results. We also compare the results with a language model to as-Bastien Moysset A2iA SA, sess if linguistic constraints do level the performance of the different networks.Our results show that for challenging datasets, 2D-LSTM networks still seem to provide the highest performances and we propose a visualization strategy to explain it.

Recurrence-free unconstrained handwritten text recognition using gated fully convolutional network

“…Training time Prediction time Parameters (min/epoch) (ms/sample) 2D-LSTM [6] 0.8 M 2D-LSTM-X2 [6] 3.3 M CNN + 1D-LSTM [5,6] 11 As we can see among the best models, ours is the one with the lowest number of parameters. Training time and prediction time of the CNN + 1D-LSTM [5,6] and of our model are in the same order of magnitude. This can be explained by the high number of normalization layers used in our model and its depth that counterbalance with the sequential computations of the LSTM layers.…”

“…In [5], a 1D-LSTM reaches better results than a 2D-LSTM with less training time but more parameters. More recently, a 2D-LSTM presented in [6] showed competitive prediction time and performance over many datasets such as RIMES [7], IAM [8] and more complex ones like MAURDOR [9].…”

“…Training time Prediction time Parameters (min/epoch) (ms/sample) 2D-LSTM [6] 0.8 M 2D-LSTM-X2 [6] 3.3 M CNN + 1D-LSTM [5,6] 11.25 57 9.6 M Ours 13.75 74 1.4 M Table IV: Training time, prediction time and number of parameters of different architectures seen in the previous experiments for the IAM dataset with image height of 128px, preserving the original width.…”

Neural networks and arbitrage in the VIX

“…Recently, there has been a trend in handwritten text recognition with deep neural networks to replace 2D recurrent layers with 1D, and in some cases even completely remove the recurrent layers, relying on simple feed-forward convolutional only architectures. A more detailed discussion of that can be found in the 2018 paper of Moysset and Messina (2017). On the other hand, those two authors show that 2D-LSTM networks still seem to provide the highest performances.…”

Boosting Offline Handwritten Text Recognition in Historical Documents With Few Labeled Lines

“…These models are configured to minimize the connectionist temporal classification (CTC) cost function proposed by Graves in [17]. In some works 2D-LSTM [16] networks are used [18]- [21]. This RNN has two main drawbacks.…”