Knowledge Injection to Neural Networks with Progressive Learning Strategy

Nguyen, Ha Thanh; Vu, Trung Kien; Racharak, Teeradaj; Nguyen, Le-Minh; Tojo, Satoshi

doi:10.1007/978-3-030-71158-0_13

Cited by 1 publication

(1 citation statement)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The final research question is answered in Chapter 5. Besides the traditional training and pretrain-finetune paradigms, there exists the third approach, knowledge injection [47]. This approach is the use of expert knowledge to support learning models and decision-making.…”

Section: Dissertation Outlinementioning

confidence: 99%

Toward Improving Attentive Neural Networks in Legal Text Processing

Nguyen¹

2022

Preprint

View full text Add to dashboard Cite

In recent years, thanks to breakthroughs in neural network techniques especially attentive deep learning models, natural language processing has made many impressive achievements. However, automated legal word processing is still a difficult branch of natural language processing. Legal sentences are often long and contain complicated legal terminologies. Hence, models that work well on general documents still face challenges in dealing with legal documents. We have verified the existence of this problem with our experiments in this work. In this dissertation, we selectively present the main achievements in improving attentive neural networks in automatic legal document processing. Language models tend to grow larger and larger, though, without expert knowledge, these models can still fail in domain adaptation, especially for specialized fields like law.This dissertation has three main tasks to achieve the goal of improving attentive models in legal document processing. First, we survey and verify the factors affecting the performance of the models when operating on a specific domain such as law. This investigation is to provide clearer insights to improve models in this domain. Second, as pretrained language models are recently the most well-known attentive approaches in natural language processing, we provide methods to create language models specific to the legal domain, producing state-of-the-art results on reliable datasets. These models are built on features from the data of legal documents, with the goal of overcoming the challenges found in our previous survey. Third, besides the approach to let the model learn completely from raw data, we propose and prove the effectiveness of using different knowledge sources to inject into the model in different ways to adjust their output. This approach not only increases explainability but also allows humans to control pretrained language models and take advantage of the knowledge resources available during the development of the field such as vocabulary, grammar, logic and law.

show abstract

Section: Dissertation Outlinementioning

confidence: 99%

Toward Improving Attentive Neural Networks in Legal Text Processing

Nguyen¹

2022

Preprint

View full text Add to dashboard Cite

show abstract

Knowledge Injection to Neural Networks with Progressive Learning Strategy

Cited by 1 publication

References 15 publications

Toward Improving Attentive Neural Networks in Legal Text Processing

Toward Improving Attentive Neural Networks in Legal Text Processing

Contact Info

Product

Resources

About