Analyzable Legal Yes/No Question Answering System using Linguistic Structures

Kano, Yoshinobu; Hoshino, Reina; Taniguchi, Ryosuke

doi:10.29007/16q5

Cited by 4 publications

(5 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Legal question answering has been the focal point of many studies. Early systems adopted rules and other unsupervised techniques for answering Yes/No questions from bar exams [66], [67]. Later works are generally supervised, using classical machine learning methods (e.g., logistic regression) [68] and different types of neural networks (e.g., CNN) [69], [70].…”

Section: A An Overview Of Major Legal Nlp Tasksmentioning

confidence: 99%

On the Effectiveness of Pre-Trained Language Models for Legal Natural Language Processing: An Empirical Study

Song

Gao

et al. 2022

IEEE Access

View full text Add to dashboard Cite

We present the first comprehensive empirical evaluation of pre-trained language models (PLMs) for legal natural language processing (NLP) in order to examine their effectiveness in this domain.Our study covers eight representative and challenging legal datasets, ranging from 900 to 57K samples, across five NLP tasks: binary classification, multi-label classification, multiple choice question answering, summarization and information retrieval. We first run unsupervised, classical machine learning and/or non-PLM based deep learning methods on these datasets, and show that baseline systems' performance can be 4%∼35% lower than that of PLM-based methods. Next, we compare general-domain PLMs and those specifically pre-trained for the legal domain, and find that domain-specific PLMs demonstrate 1%∼5% higher performance than general-domain models, but only when the datasets are extremely close to the pretraining corpora. Finally, we evaluate six general-domain state-of-the-art systems, and show that they have limited generalizability to legal data, with performance gains from 0.1% to 1.2% over other PLM-based methods. Our experiments suggest that both general-domain and domain-specific PLM-based methods generally achieve better results than simpler methods on most tasks, with the exception of the retrieval task, where the best-performing baseline outperformed all PLM-based methods by at least 5%. Our findings can help legal NLP practitioners choose the appropriate methods for different tasks, and also shed light on potential future directions for legal NLP research.

show abstract

Section: A An Overview Of Major Legal Nlp Tasksmentioning

confidence: 99%

On the Effectiveness of Pre-Trained Language Models for Legal Natural Language Processing: An Empirical Study

Song

Gao

et al. 2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Kano (2016) suggested a penalized scoring method assigning scores to parts of documents that include terms, which indicate that the answer is no. Kano, Hoshino, and Taniguchi (2017) built a system using linguistic analysis to find correspondences of predicates and arguments from the given problem sentences and knowledge source sentences. Although these approaches are interpretable, the experience has shown that more complex systems are necessary for solving the QA problem, while the machine learning approaches are necessary for generalization purposes.…”

Section: Related Workmentioning

confidence: 99%

Artificial fine-tuning tasks for yes/no question answering

Dimitriadis

Tsoumakas

2022

Nat. Lang. Eng.

View full text Add to dashboard Cite

Current research in yes/no question answering (QA) focuses on transfer learning techniques and transformer-based models. Models trained on large corpora are fine-tuned on tasks similar to yes/no QA, and then the captured knowledge is transferred for solving the yes/no QA task. Most previous studies use existing similar tasks, such as natural language inference or extractive QA, for the fine-tuning step. This paper follows a different perspective, hypothesizing that an artificial yes/no task can transfer useful knowledge for improving the performance of yes/no QA. We introduce three such tasks for this purpose, by adapting three corresponding existing tasks: candidate answer validation, sentiment classification, and lexical simplification. Furthermore, we experimented with three different variations of the BERT model (BERT base, RoBERTa, and ALBERT). The results show that our hypothesis holds true for all artificial tasks, despite the small size of the corresponding datasets that are used for the fine-tuning process, the differences between these tasks, the decisions that we made to adapt the original ones, and the tasks’ simplicity. This gives an alternative perspective on how to deal with the yes/no QA problem, that is more creative, and at the same time more flexible, as it can exploit multiple other existing tasks and corresponding datasets to improve yes/no QA models.

show abstract

“…Works based on this approach, such as (Kim et al, 2013) have developed different approaches to answer yes/no questions relevant to civil laws in legal bar exams. A bar examination is intended Work Approach (Kim et al, 2013) Antonym detection and Semantic Analysis (Kim et al, 2014) TF-IDF, LDA and SVM (Kim et al, 2016) Paraphrasing detection (Taniguchi & Kano, 2016) Case-role analysis (Kano et al, 2017) Using linguistic structures (Taniguchi et al, 2018) Using Framenet to determine whether a candidate is qualified to practice law in a given jurisdiction. There is a recurring concern in the literature on whether it is possible to pass this type of test without human supervision.…”

Section: Yes/no Answersmentioning

confidence: 99%

“…Other authors (Taniguchi & Kano, 2016;Taniguchi et al, 2018) has addressed the problem by case-role analysis and Framenet 5 , respectively. Finally, Kano et al has explored, for the first time, linguistic structures (Kano et al, 2017). Table 3 summarizes all these works.…”

Section: Yes/no Answersmentioning

confidence: 99%

A Survey on Legal Question Answering Systems

Martinez‐Gil¹

2021

Preprint

View full text Add to dashboard Cite

Many legal professionals think that the explosion of information about local, regional, national, and international legislation makes their practice more costly, time-consuming, and even error-prone. The two main reasons for this are that most legislation is usually unstructured, and the tremendous amount and pace with which laws are released causes information overload in their daily tasks. In the case of the legal domain, the research community agrees that a system allowing to generate automatic responses to legal questions could substantially impact many practical implications in daily activities.The degree of usefulness is such that even a semi-automatic solution could significantly help to reduce the workload to be faced. This is mainly because a Question Answering system could be able to automatically process a massive amount of legal resources to answer a question or doubt in seconds, which means that it could save resources in the form of effort, money, and time to many professionals in the legal sector. In this work, we quantitatively and qualitatively survey the solutions that currently exist to meet this challenge.

show abstract

Analyzable Legal Yes/No Question Answering System using Linguistic Structures

Cited by 4 publications

References 19 publications

On the Effectiveness of Pre-Trained Language Models for Legal Natural Language Processing: An Empirical Study

On the Effectiveness of Pre-Trained Language Models for Legal Natural Language Processing: An Empirical Study

Artificial fine-tuning tasks for yes/no question answering

A Survey on Legal Question Answering Systems

Contact Info

Product

Resources

About