Character-Based Embedding Models and Reranking Strategies for Understanding Natural Language Meal Descriptions

Korpusik, Mandy; Collins, Zachary; Glass, James

doi:10.21437/interspeech.2017-422

Cited by 9 publications

(3 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…String transduction tries to map one string to another and can be used for misspelled typo corrections [13]. Machine learning is used in character scale to typo detection and corrections, but the recall rate is low (about 30%) [14].…”

Section: Literature Reviewmentioning

confidence: 99%

New labeled dataset of interconnected lexical typos for automatic correction in the bug reports

Neysiani

Babamir

2019

SN Appl. Sci.

View full text Add to dashboard Cite

Large-scale and especially open-source projects use software triage systems like Bugzilla to manage their user's requests like bugs, suggestions, and requirements. The software triage systems have many tasks like prioritizing, finding duplicate and assigning bug reports to developers automatically, which needs text mining, information retrieval, and natural language processing techniques. We already showed there are many typos in the bug reports which reduce the performance of artificial intelligence techniques. The connected terms were one of the most types of typos in the context of bug reports. Also, we introduce some algorithms to correct the connected terms earlier, but there was not any labeled dataset that can be used to evaluate the accuracy of process of typo correction. Now we made a new labeled dataset including 42,970 typos between 182,096 to can be used for the typo correction evaluation process. There are 52% connected typos in the labeled dataset, which show the previous results about the number of connected typos were correct. Then we used the typo correction algorithms which were introduced in prior studies to evaluate their accuracy. The experimental results show 81.6% and 83.3% accuracy in top-5 and top-10 suggestions of the list of typo corrections, respectively.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

New labeled dataset of interconnected lexical typos for automatic correction in the bug reports

Neysiani

Babamir

2019

SN Appl. Sci.

View full text Add to dashboard Cite

show abstract

“…details of which are given in (Gormley and Tong, 2015), we wanted a method that could be fine-tuned to our dataset without creating hand-crafted rules. Recent progress in creating distributional representations of words has found applications in Information Retrieval (IR) due to work by Zamani and Croft (2017), Korpusik et al (2017) and Cao and Lu (2017) among others. Some of these models provide the query as an input to a neural network at prediction time while others rely on using the embeddings for query expansion.…”

Section: Previous Workmentioning

confidence: 99%

Proceedings of the Second Workshop on Subword/Character LEvel Models

2018

View full text Add to dashboard Cite

ii Introduction Traditional NLP starts with a hand-engineered layer of representation, the level of tokens or words. A tokenization component first breaks up the text into units using manually designed rules. Tokens are then processed by components such as word segmentation, morphological analysis and multiword recognition. The heterogeneity of these components makes it hard to create integrated models of both structure within tokens (e.g., morphology) and structure across multiple tokens (e.g., multi-word expressions). This approach can perform poorly (i) for morphologically rich languages, (ii) for noisy text, (iii) for languages in which the recognition of words is difficult and (iv) for adaptation to new domains; and (v) it can impede the optimization of preprocessing in end-to-end learning.The workshop provides a forum for discussing recent advances as well as future directions on sub-word and character-level natural language processing and representation learning that address these problems. Topics of Interest:• tokenization-free models• character-level machine translation• character-ngram information retrieval• transfer learning for character-level models• models of within-token and cross-token structure• NL generation (of words not seen in training etc)• out of vocabulary words• morphology and segmentation• relationship b/w morphology and character-level models• stemming and lemmatization • inflection generation• orthographic productivity• form-meaning representations• true end-to-end learning AbstractNeural machine translation has achieved impressive results in the last few years, but its success has been limited to settings with large amounts of parallel data. One way to improve NMT for lower-resource settings is to initialize a word-based NMT model with pretrained word embeddings. However, rare words still suffer from lower quality word embeddings when trained with standard word-level objectives. We introduce word embeddings that utilize morphological resources, and compare to purely unsupervised alternatives. We work with Arabic, a morphologically rich language with available linguistic resources, and perform Ar-to-En MT experiments on a small corpus of TED subtitles. We find that word embeddings utilizing subword information consistently outperform standard word embeddings on a word similarity task and as initialization of the source word embeddings in a low-resource NMT system.

show abstract

“…In our prior work, we explored convolutional neural network (CNN) models for semantic tagging and mapping of natural language meal descriptions to a structured food database [1,2,3], as well as for dialogue state tracking [4,5,6]. In this work, we demonstrate that our CNN generalizes to other domains beyond nutrition, outperforming prior state-of-the-art on the benchmark ATIS task, as well as a restaurant query dataset.…”

Section: Introductionmentioning

confidence: 97%

A Comparison of Deep Learning Methods for Language Understanding

Korpusik

Liu²,

Glass

2019

Interspeech 2019

Self Cite

View full text Add to dashboard Cite

In this paper, we compare a suite of neural networks (recurrent, convolutional, and the recently proposed BERT model) to a CRF with hand-crafted features on three semantic tagging corpora: the Air Travel Information System (ATIS) benchmark, restaurant queries, and written and spoken meal descriptions. Our motivation is to investigate pre-trained BERT's transferability to the domains we are interested in. We demonstrate that neural networks without feature engineering outperform state-of-the-art statistical and deep learning approaches on all three tasks (except written meal descriptions, where the CRF is slightly better) and that deep, attention-based BERT, in particular, surpasses state-of-the-art results on these tasks. Error analysis shows the models are less confident when making errors, enabling the system to follow up with the user when uncertain.

show abstract

Character-Based Embedding Models and Reranking Strategies for Understanding Natural Language Meal Descriptions

Cited by 9 publications

References 14 publications

New labeled dataset of interconnected lexical typos for automatic correction in the bug reports

New labeled dataset of interconnected lexical typos for automatic correction in the bug reports

Proceedings of the Second Workshop on Subword/Character LEvel Models

A Comparison of Deep Learning Methods for Language Understanding

Contact Info

Product

Resources

About