Recurrent neural network-based models for recognizing requisite and effectuation parts in legal texts

Nguyen, T. D.; Nguyen, Le-Minh; Tojo, Satoshi; Satoh, Ken; Shimazu, Akira

doi:10.1007/s10506-018-9225-1

Cited by 39 publications

(13 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, “Microsoft” and “Google” have similar semantics, both being tech companies. The words “car” and “journey” are not semantically similar, but these two words are related, and both are related to transportation [ 14 , 15 ].…”

Section: Introductionmentioning

confidence: 99%

Application of Adaptive Neural Network Algorithm Model in English Text Analysis

Hung

Hsiao

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Based on the existing optimization neural network algorithm, this paper introduces a simple and computationally efficient adaptive mechanism (adaptive exponential decay rate). By applying the adaptive mechanism to the Adadelta algorithm, it can be seen that AEDR-Adadelta acquires the learning rate dynamically and adaptively. At the same time, by proposing an adaptive exponential decay rate, the number and method of configuring hyperparameters can be reduced, and different learning rates can be effectively obtained for different parameters. The model is based on the encoder-decoder structure and adopts a dual-encoder structure. The transformer encoder is used to extract the context information of the sentence; the Bi-GRU encoder is used to extract the information of the source sentence; and the gated structure is used at the decoder side. The input information is integrated, and each part is matched with different attention mechanisms, which improves the model’s ability to extract and analyze relevant features in sentences. In order to accurately capture the coherence features in English texts, an improved subgraph matching algorithm is used to mine frequently occurring subgraph patterns in sentence semantic graphs, which are used to simulate the unique coherence patterns in English texts, and then analyze the overall coherence of English texts. According to the frequency of occurrence of different subgraph patterns in the sentence semantic graph, the subgraphs are filtered to generate frequent subgraph sets, and the subgraph frequency of each frequent subgraph is calculated separately. The overall coherence quality of English text is quantitatively analyzed by extracting the distribution characteristics of frequent subgraphs and the semantic values of subgraphs in the sentence semantic graph. According to the experimental results, the algorithm using the adaptive mechanism can reduce the error of the training set and the test set, improve the classification accuracy to a certain extent, and has a faster convergence speed and better text generalization ability. The semantic coherence diagnosis model of English text in this paper performs well in various tasks and has a good effect on improving the automatic correction system of English composition and providing reference for English teachers’ composition correction.

show abstract

Section: Introductionmentioning

confidence: 99%

Application of Adaptive Neural Network Algorithm Model in English Text Analysis

Hung

Hsiao

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

show abstract

“…Therefore, it is difficult for a language model with averaging and interpolation capabilities to infer logical structures on its own through unsupervised training. To correctly annotate law sentences with many interlocking logical structures, we need to use multilayer annotation [49]. Table 5.2 is the annotation of the above example.…”

Section: Methodsmentioning

confidence: 99%

Toward Improving Attentive Neural Networks in Legal Text Processing

Nguyen¹

2022

Preprint

View full text Add to dashboard Cite

In recent years, thanks to breakthroughs in neural network techniques especially attentive deep learning models, natural language processing has made many impressive achievements. However, automated legal word processing is still a difficult branch of natural language processing. Legal sentences are often long and contain complicated legal terminologies. Hence, models that work well on general documents still face challenges in dealing with legal documents. We have verified the existence of this problem with our experiments in this work. In this dissertation, we selectively present the main achievements in improving attentive neural networks in automatic legal document processing. Language models tend to grow larger and larger, though, without expert knowledge, these models can still fail in domain adaptation, especially for specialized fields like law.This dissertation has three main tasks to achieve the goal of improving attentive models in legal document processing. First, we survey and verify the factors affecting the performance of the models when operating on a specific domain such as law. This investigation is to provide clearer insights to improve models in this domain. Second, as pretrained language models are recently the most well-known attentive approaches in natural language processing, we provide methods to create language models specific to the legal domain, producing state-of-the-art results on reliable datasets. These models are built on features from the data of legal documents, with the goal of overcoming the challenges found in our previous survey. Third, besides the approach to let the model learn completely from raw data, we propose and prove the effectiveness of using different knowledge sources to inject into the model in different ways to adjust their output. This approach not only increases explainability but also allows humans to control pretrained language models and take advantage of the knowledge resources available during the development of the field such as vocabulary, grammar, logic and law.

show abstract

“…While no absolute winner was observed, the study highlights the benefit of using feature weights or network attention weights from these predictive models to identify salient phrases in motions or contentions and case facts. Nguyen et al (2017Nguyen et al ( , 2018 propose several approaches to train long short term memory (LSTMs) models and conditional random field (CRF) models for the problem of identifying two key portions of legal documents, i.e., requisite and effectuation segments, with evaluation on Japanese civil code and Japanese National Pension Law dataset. In Chalkidis and Kampas (2019) by Chalkidis and Kampas, a major contribution is the development of word2vec skip-gram embeddings trained on large legal corpora (mostly from European, UK, and US legislations).…”

Section: Related Workmentioning

confidence: 99%

Unsupervised law article mining based on deep pre-trained language representation models with application to the Italian civil code

Tagarelli

Simeri

2021

Artif Intell Law

View full text Add to dashboard Cite

Modeling law search and retrieval as prediction problems has recently emerged as a predominant approach in law intelligence. Focusing on the law article retrieval task, we present a deep learning framework named LamBERTa, which is designed for civil-law codes, and specifically trained on the Italian civil code. To our knowledge, this is the first study proposing an advanced approach to law article prediction for the Italian legal system based on a BERT (Bidirectional Encoder Representations from Transformers) learning framework, which has recently attracted increased attention among deep learning approaches, showing outstanding effectiveness in several natural language processing and learning tasks. We define LamBERTa models by fine-tuning an Italian pre-trained BERT on the Italian civil code or its portions, for law article retrieval as a classification task. One key aspect of our LamBERTa framework is that we conceived it to address an extreme classification scenario, which is characterized by a high number of classes, the few-shot learning problem, and the lack of test query benchmarks for Italian legal prediction tasks. To solve such issues, we define different methods for the unsupervised labeling of the law articles, which can in principle be applied to any law article code system. We provide insights into the explainability and interpretability of our LamBERTa models, and we present an extensive experimental analysis over query sets of different type, for single-label as well as multi-label evaluation tasks. Empirical evidence has shown the effectiveness of LamBERTa, and also its superiority against widely used deep-learning text classifiers and a few-shot learner conceived for an attribute-aware prediction task.

show abstract

Recurrent neural network-based models for recognizing requisite and effectuation parts in legal texts

Cited by 39 publications

References 16 publications

Application of Adaptive Neural Network Algorithm Model in English Text Analysis

Application of Adaptive Neural Network Algorithm Model in English Text Analysis

Toward Improving Attentive Neural Networks in Legal Text Processing

Unsupervised law article mining based on deep pre-trained language representation models with application to the Italian civil code

Contact Info

Product

Resources

About