Fast and Accurate Decision Trees for Natural Language Processing Tasks

Boroş, Tiberiu; Dumitrescu, Ştefan Daniel; Pipa, Sonia

doi:10.26615/978-954-452-049-6_016

Cited by 13 publications

(3 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Traditionally, natural language processing (NLP) frameworks were based on white-box methods such as rule-based systems (Allen, 1988;Ribeiro et al, 2019;Ribeiro and Forbus, 2021) and decision trees (Boros et al, 2017), which were inherently inspectable (Danilevsky et al, 2020). More recently, large deep learning language models (black-box methods) have gained popularity (Song et al, 2020;Raffel et al, 2020), but their improvements in result quality came with a cost: the system's outputs lack explainability and inspectability.…”

Section: Related Workmentioning

confidence: 99%

Entailment Tree Explanations via Iterative Retrieval-Generation Reasoner

Ribeiro¹,

Wang²,

Ma³

et al. 2022

Findings of the Association for Computational Linguistics: NAACL 2022

View full text Add to dashboard Cite

Large language models have achieved high performance on various question answering (QA) benchmarks, but the explainability of their output remains elusive. Structured explanations, called entailment trees, were recently suggested as a way to explain and inspect a QA system's answer. In order to better generate such entailment trees, we propose an architecture called Iterative Retrieval-Generation Reasoner (IRGR). Our model is able to explain a given hypothesis by systematically generating a stepby-step explanation from textual premises. The IRGR model iteratively searches for suitable premises, constructing a single entailment step at a time. Contrary to previous approaches, our method combines generation steps and retrieval of premises, allowing the model to leverage intermediate conclusions, and mitigating the input size limit of baseline encoder-decoder models. We conduct experiments using the EN-TAILMENTBANK dataset, where we outperform existing benchmarks on premise retrieval and entailment tree generation, with around 300% gain in overall correctness.

show abstract

Section: Related Workmentioning

confidence: 99%

Entailment Tree Explanations via Iterative Retrieval-Generation Reasoner

Ribeiro¹,

Wang²,

Ma³

et al. 2022

Findings of the Association for Computational Linguistics: NAACL 2022

View full text Add to dashboard Cite

show abstract

“…Global vs. Local. Rule-based approaches (Hearst 1992;Brin 1998) or decision trees (Béchet, Nasr, and Genet 2000;Boros, Dumitrescu, and Pipa 2017) provide global explainability by constructing transparent models that people can understand. However, these directions were slowly replaced by deep learning, which tends to yield better classifiers (at least with respect to accuracy).…”

Section: A Taxonomy Of Explanationsmentioning

confidence: 99%

It Takes Two Flints to Make a Fire: Multitask Learning of Neural Relation and Explanation Classifiers

Tang¹,

Surdeanu²

2022

Preprint

View full text Add to dashboard Cite

We propose an explainable approach for relation extraction that mitigates the tension between generalization and explainability by jointly training for the two goals. Our approach uses a multi-task learning architecture, which jointly trains a classifier for relation extraction, and a sequence model that labels words in the context of the relation that explain the decisions of the relation classifier. We also convert the model outputs to rules to bring global explanations to this approach. This sequence model is trained using a hybrid strategy: supervised, when supervision from pre-existing patterns is available, and semi-supervised otherwise. In the latter situation, we treat the sequence model's labels as latent variables, and learn the best assignment that maximizes the performance of the relation classifier. We evaluate the proposed approach on the two datasets and show that the sequence model provides labels that serve as accurate explanations for the relation classifier's decisions, and, importantly, that the joint training generally improves the performance of the relation classifier. We also evaluate the performance of the generated rules and show that the new rules are great add-on to the manual rules and bring the rule-based system much closer to the neural models.

show abstract

“…Theconsiderableeffortofanaphoraresolutiononlymakessenseifitinfluencestheperformance ofinformationretrievalorlanguageprocessingsystems.Therearestudiesontheimpactofanaphora resolutionontheperformanceofretrievalsystemsorneighboringapproachesasquestionanswering systemsortextsummarization.ResearchersfromSyracuseUniversityconductedexperimentson anaphoraresolutioninabstractsofscientificarticles (Bonzi,1991;DuRossLiddy,1990;Liddy,Bonzi, Katzer,&Oddy,1987), Orasan(2007)analyzedanaphoraresolutionforoptimizingtextsummarization, VicedoandFerrández(2000wereabletoshowtheimportanceofpronominalanaphoraresolutionin questionansweringsystems,and,finally, Pirkola(1996)provedtherelevanceofanaphoraresolution forsearcheswithproximityoperators.…”

Section: Introductionmentioning

confidence: 99%

Anaphora Resolution

Gros

Habermann

Kirstein

et al. 2018

International Journal of Information Retrieval Research

View full text Add to dashboard Cite

This article analyses the effect of anaphora resolution on information retrieval performance for systems with relevance ranking. It will be investigated if the Mean Average Precision of a retrieval system is improved after an intellectual replacement of all anaphors in a corpus with various texts. These texts mostly consist of news stories and fairy tales, thus covering two varying genres with different amounts of anaphors. A model retrieval system is developed using Lucene to measure the effects of anaphora resolution. Different queries are used and the rankings are analysed in order to show the changes induced by the anaphora resolution. In addition, approaches of automated anaphora resolution are considered. It turns out that the Mean Average Precision improves noticeably by 36% after the anaphora resolution. Thus, it is highly recommended to improve existing approaches of automated anaphora resolution in the future as current attempts do not yet yield satisfying results.

show abstract

Fast and Accurate Decision Trees for Natural Language Processing Tasks

Cited by 13 publications

References 13 publications

Entailment Tree Explanations via Iterative Retrieval-Generation Reasoner

Entailment Tree Explanations via Iterative Retrieval-Generation Reasoner

It Takes Two Flints to Make a Fire: Multitask Learning of Neural Relation and Explanation Classifiers

Anaphora Resolution

Contact Info

Product

Resources

About