Lawformer: A pre-trained language model for Chinese legal long documents

Xiao, Chaojun; Hu, Xueyu; Liu, Zhiyuan; Tu, Cunchao; Sun, Maosong

doi:10.1016/j.aiopen.2021.06.003

Cited by 107 publications

(72 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…LegalAI can not only provide handy references for people who are not familiar with legal knowledge, but also reduce the redundant paperwork for legal practitioners. Many efforts have been devoted to a variety of LegalAI tasks, including legal judgment prediction Chalkidis et al, 2019;Yang et al, 2019), legal question answering (Ravichander et al, 2019;Zhong et al, 2020b;Kien et al, 2020), contract review (Hendrycks et al, 2021;Koreeda and Manning, 2021), legal case retrieval , and legal pre-trained models (Chalkidis et al, 2020;Xiao et al, 2021). Most existing works focus on the application in LegalAI while ignoring the basic key event information in the legal documents.…”

Section: Legal Artificial Intelligencementioning

confidence: 99%

“…As pre-trained language models have achieved promising results in many legal tasks (Chalkidis et al, 2020;Xiao et al, 2021), we adopt the BERT as our basic encoder. To verify the effectiveness of event detection in LegalAI, we only make minor changes in the embedding layer to integrate the event information.…”

Section: Encoder Architecturementioning

confidence: 99%

“…The training data for the low-resource setting is obtained by randomly selecting 50 samples for each label and the corresponding data we used is also released in the github repository. The experiments are based on the source code released in Zhong et al (2020a For Similar Case Retrieval (SCR), we use LeCaRD 7 as benchmark and implement the models based on the code released in Xiao et al (2021). As the documents in LeCaRD are usually longer than 512, we truncate the text to feed into the encoder.…”

Section: A2 Downstream Applicationsmentioning

confidence: 99%

See 2 more Smart Citations

LEVEN: A Large-Scale Chinese Legal Event Detection Dataset

Feng¹,

Xiao²,

Wang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Recognizing facts is the most fundamental step in making judgments, hence detecting events in the legal documents is important to legal case analysis tasks. However, existing Legal Event Detection (LED) datasets only concern incomprehensive event types and have limited annotated data, which restricts the development of LED methods and their downstream applications. To alleviate these issues, we present LEVEN, a large-scale Chinese LEgal eVENt detection dataset, with 8, 116 legal documents and 150, 977 humanannotated event mentions in 108 event types. Not only charge-related events, LEVEN also covers general events, which are critical for legal case understanding but neglected in existing LED datasets. To our knowledge, LEVEN is the largest LED dataset and has dozens of times the data scale of others, which shall significantly promote the training and evaluation of LED methods. The results of extensive experiments indicate that LED is challenging and needs further effort. Moreover, we simply utilize legal events as side information to promote downstream applications. The method achieves improvements of average 2.2 points precision in low-resource judgment prediction, and 1.5 points mean average precision in unsupervised case retrieval, which suggests the fundamentality of LED. The source code and dataset can be obtained from https: //github.com/thunlp/LEVEN.

show abstract

Section: Legal Artificial Intelligencementioning

confidence: 99%

Section: Encoder Architecturementioning

confidence: 99%

Section: A2 Downstream Applicationsmentioning

confidence: 99%

See 1 more Smart Citation

LEVEN: A Large-Scale Chinese Legal Event Detection Dataset

Feng¹,

Xiao²,

Wang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Charges r Prison terms r Link TOPJUDGE-CJO [29] -TOPJUDGE-PKU [29] TOPJUDGE-CAIL [29] CAIL-Long [47] Link FLA [15] Charges r -RACP [48] Fact snippets a MAMD [49] -Court-View-Gen [33] Court View r Link AC-NLG [34] Facts r…”

Section: Articles Rmentioning

confidence: 99%

A Survey on Legal Judgment Prediction: Datasets, Metrics, Models and Challenges

Cui¹,

Shen²,

Nie³

et al. 2022

Preprint

View full text Add to dashboard Cite

Legal judgment prediction (LJP) applies Natural Language Processing (NLP) techniques to predict judgment results based on fact descriptions automatically. Recently, large-scale public datasets and advances in NLP research have led to increasing interest in LJP. Despite a clear gap between machine and human performance, impressive results have been achieved in various benchmark datasets. In this paper, to address the current lack of comprehensive survey of existing LJP tasks, datasets, models and evaluations, (1) we analyze 31 LJP datasets in 6 languages, present their construction process and define a classification method of LJP with 3 different attributes;(2) we summarize 14 evaluation metrics under four categories for different outputs of LJP tasks; (3) we review 12 legal-domain pretrained models in 3 languages and highlight 3 major research directions for LJP; (4) we show the state-of-art results for 8 representative datasets from different court cases and discuss the open challenges. This paper can provide up-to-date and comprehensive reviews to help readers understand the status of LJP. We hope to facilitate both NLP researchers and legal professionals for further joint efforts in this problem.

show abstract

“…Although BERT-based methods have made breakthroughs in different fields in recent years [3][4][5][6][7][8], there is still a deficiency in terms of solving understanding long, domain-specific text, mainly because pre-trained models trained by generic corpora often do not work well in the legal field. There are currently several pre-trained models for the legal domain [9][10][11]. Nevertheless, there is only one open-source pre-training model for Chinese judicial scripts and its effect has been experimentally verified to have no significant performance improvement.…”

Section: Introductionmentioning

confidence: 99%

Legal Judgment Prediction via Heterogeneous Graphs and Knowledge of Law Articles

Zhao

Zhou

et al. 2022

Applied Sciences

View full text Add to dashboard Cite

Legal judgment prediction (LJP) is a crucial task in legal intelligence to predict charges, law articles and terms of penalties based on case fact description texts. Although existing methods perform well, they still have many shortcomings. First, the existing methods have significant limitations in understanding long documents, especially those based on RNNs and BERT. Secondly, the existing methods are not good at solving the problem of similar charges and do not fully and effectively integrate the information of law articles. To address the above problems, we propose a novel LJP method. Firstly, we improve the model’s comprehension of the whole document based on a graph neural network approach. Then, we design a graph attention network-based law article distinction extractor to distinguish similar law articles. Finally, we design a graph fusion method to fuse heterogeneous graphs of text and external knowledge (law article group distinction information). The experiments show that the method could effectively improve LJP performance. The experimental metrics are superior to the existing state of the art.

show abstract

Lawformer: A pre-trained language model for Chinese legal long documents

Cited by 107 publications

References 24 publications

LEVEN: A Large-Scale Chinese Legal Event Detection Dataset

LEVEN: A Large-Scale Chinese Legal Event Detection Dataset

A Survey on Legal Judgment Prediction: Datasets, Metrics, Models and Challenges

Legal Judgment Prediction via Heterogeneous Graphs and Knowledge of Law Articles

Contact Info

Product

Resources

About