Regular expressions for decoding of neural network outputs

Strauß, Tobias; Leifert, Gundram; Grüning, Tobias; Labahn, Roger

doi:10.1016/j.neunet.2016.03.003

Cited by 6 publications

(3 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since 2009, tremendous progress in the field of Automated Text Recognition 1 (ATR) [4,5] as well as Keyword Spotting (KWS) [6,7,8] was achieved. The performance of state-of-the-art systems reaches character error rates below 10% for ATR [9] and mean average precisions above 0.9 for KWS [10] for complex handwritten documents.…”

mentioning

confidence: 99%

A two-stage method for text line detection in historical documents

Grüning

Leifert

Strauß

et al. 2019

IJDAR

Self Cite

108

View full text Add to dashboard Cite

This work presents a two-stage text line detection method for historical documents. Each detected text line is represented by its baseline. In a first stage, a deep neural network called ARU-Net labels pixels to belong to one of the three classes: baseline, separator or other. The separator class marks beginning and end of each text line. The ARU-Net is trainable from scratch with manageably few manually annotated example images (less than 50). This is achieved by utilizing data augmentation strategies. The network predictions are used as input for the second stage which performs a bottom-up clustering to build baselines. The developed method is capable of handling complex layouts as well as curved and arbitrarily oriented text lines. It substantially outperforms current state-of-the-art approaches. For example, for the complex track of the cBAD: IC-DAR2017 Competition on Baseline Detection the Fvalue is increased from 0.859 to 0.922. The framework to train and run the ARU-Net is open source.

show abstract

mentioning

confidence: 99%

A two-stage method for text line detection in historical documents

Grüning

Leifert

Strauß

et al. 2019

IJDAR

Self Cite

108

View full text Add to dashboard Cite

show abstract

“…NNs and REs. As for NNs and REs, previous work has tried to use RE to speed up the decoding phase of a NN (Strauß et al, 2016) and generating REs from natural language specifications of the 7 We do not include results of both for slot filling since its REs are different from feat and logit, and we have already shown that the attention loss method does not work for slot filling.…”

Section: Related Workmentioning

confidence: 99%

Marrying Up Regular Expressions with Neural Networks: A Case Study for Spoken Language Understanding

Luo¹,

Feng²,

Wang³

et al. 2018

Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

View full text Add to dashboard Cite

The success of many natural language processing (NLP) tasks is bound by the number and quality of annotated data, but there is often a shortage of such training data. In this paper, we ask the question: "Can we combine a neural network (NN) with regular expressions (RE) to improve supervised learning for NLP?". In answer, we develop novel methods to exploit the rich expressiveness of REs at different levels within a NN, showing that the combination significantly enhances the learning effectiveness when a small number of training examples are available. We evaluate our approach by applying it to spoken language understanding for intent detection and slot filling. Experimental results show that our approach is highly effective in exploiting the available training data, giving a clear boost to the RE-unaware NN. flights from Boston to Miami Intent RE: Intent Label: flight /from (__CITY) to (__CITY)/ O O B-fromloc.city O B-toloc.city Sentence: Slot Labels: Slot RE: /^flights? from/ REtag: flight city / toloc.city REtag: city / fromloc.city

show abstract

“…Rules are expressions of knowledge and regular expression is a kind of rules involving a lot of knowledge for specific domains. Strauß et al (2016) build a decoder based on REs to speed up the decoding of NNs. Li et al (2017) encode semantic features into CNNs filters instead of initializing them randomly, which helps filters focus on learning useful n-grams.…”

Section: Related Workmentioning

confidence: 99%

Generalize Symbolic Knowledge With Neural Rule Engine

Shen¹,

Xu²,

Lu³

2018

Preprint

View full text Add to dashboard Cite

Neural-symbolic learning aims to take the advantages of both neural networks and symbolic knowledge to build better intelligent systems. As neural networks have dominated the state-of-the-art results in a wide range of NLP tasks, it attracts considerable attention to improve the performance of neural models by integrating symbolic knowledge. Different from existing works, this paper investigates the combination of these two powerful paradigms from the knowledge-driven side. We propose Neural Rule Engine (NRE), which can learn knowledge explicitly from logic rules and then generalize them implicitly with neural networks. NRE is implemented with neural module networks in which each module represents an action of the logic rule. The experiments show that NRE could greatly improve the generalization abilities of logic rules with a significant increase on recall. Meanwhile, the precision is still maintained at a high level.

show abstract

Regular expressions for decoding of neural network outputs

Cited by 6 publications

References 16 publications

A two-stage method for text line detection in historical documents

A two-stage method for text line detection in historical documents

Marrying Up Regular Expressions with Neural Networks: A Case Study for Spoken Language Understanding

Generalize Symbolic Knowledge With Neural Rule Engine

Contact Info

Product

Resources

About