LEGAL-BERT: The Muppets straight out of Law School

Chalkidis, Ilias; Fergadiotis, Manos; Malakasiotis, Prodromos; Αλέτρας, Νικόλαος; Androutsopoulos, Ion

doi:10.18653/v1/2020.findings-emnlp.261

Cited by 317 publications

(184 citation statements)

References 19 publications

(17 reference statements)

Supporting

Mentioning

118

Contrasting

Unclassified

Order By: Relevance

“…This evidence comes from the court's own guidelines (ECtHR, 2014), but can also be found in the writings of a former judge of the ECtHR (Zupancic, 2016) and of legal scholars (Lupu and Voeten, 2010). Second, there is existing research on the neural modeling of ECtHR case law we can build upon (Aletras et al, 2016;Chalkidis et al, 2019Chalkidis et al, , 2020. Third, the documents of the ECtHR case law, unlike those of most other courts, textually separate the facts from the arguments, which is crucial for our experiments.…”

Section: K K=1 P(o K | •)mentioning

confidence: 99%

What About the Precedent: An Information-Theoretic Analysis of Common Law

Valvoda¹,

Pimentel²,

Stoehr³

et al. 2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

In common law, the outcome of a new case is determined mostly by precedent cases, rather than by existing statutes. However, how exactly does the precedent influence the outcome of a new case? Answering this question is crucial for guaranteeing fair and consistent judicial decision-making. We are the first to approach this question computationally by comparing two longstanding jurisprudential views; Halsbury's, who believes that the arguments of the precedent are the main determinant of the outcome, and Goodhart's, who believes that what matters most is the precedent's facts. We base our study on the corpus of legal cases from the European Court of Human Rights (ECtHR), which allows us to access not only the case itself, but also cases cited in the judges' arguments (i.e. the precedent cases). Taking an information-theoretic view, and modeling the question as a case outcome classification task, we find that the precedent's arguments share 0.38 nats of information with the case's outcome, whereas precedent's facts only share 0.18 nats of information (i.e., 58% less); suggesting Halsbury's view may be more accurate in this specific court. We found however in a qualitative analysis that there are specific statues where Goodhart's view dominates, and present some evidence these are the ones where the legal concept at hand is less straightforward.

show abstract

Section: K K=1 P(o K | •)mentioning

confidence: 99%

What About the Precedent: An Information-Theoretic Analysis of Common Law

Valvoda¹,

Pimentel²,

Stoehr³

et al. 2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

show abstract

“…The BERT language model (Devlin et al, 2019) has been shown to be effective on a wide range of tasks in the legal domain, including Named Entity Recognition (Chalkidis et al, 2020), annotation of legal concepts (Chalkidis et al, 2020), and evidence retrieval (Soleimani et al, 2020).…”

Section: Approachmentioning

confidence: 99%

NLP for Consumer Protection: Battling Illegal Clauses in German Terms and Conditions in Online Shopping

Braun¹,

Matthes²

2021

Proceedings of the 1st Workshop on NLP for Positive Impact

View full text Add to dashboard Cite

Online shopping is an ever more important part of the global consumer economy, not just in times of a pandemic. When we place an order online as consumers, we regularly agree to the so-called "Terms and Conditions" (T&C), a contract unilaterally drafted by the seller. Often, consumers do not read these contracts and unwittingly agree to unfavourable and often void terms. Government and non-government organisations (NGOs) for consumer protection battle such terms on behalf of consumers, who often hesitate to take on legal actions themselves. However, the growing number of online shops and a lack of funding makes it increasingly difficult for such organisations to monitor the market effectively. This paper describes how Natural Language Processing (NLP) can be applied to support consumer advocates in their efforts to protect consumers. Together with two NGOs from Germany, we developed an NLP-based application that legally assesses clauses in T&C from German online shops under the European Union's (EU) jurisdiction. We report that we could achieve an accuracy of 0.9 in the detection of void clauses by fine-tuning a pre-trained German BERT model. The approach is currently used by two NGOs and has already helped to challenge void clauses in T&C.

show abstract

“…SciBERT took 7 days to train from scratch with a single TPU v3 with 3 cores [9]. In [18], a legal language model, LEGAL-BERT, is trained on a 12 GB corpus of legal texts, either from scratch or further pre-trained from BERT-Base. The authors found that both were valid approaches with similar results.…”

Section: B Domain-specific Language Modelsmentioning

confidence: 99%

“…The authors found that both were valid approaches with similar results. Our training corpus has a similar size as [18] and we use a single NVIDIA V100 GPU with 16 cores to train our models. Based on these limitations, the decision was taken to further pre-train our domain-specific models rather than train them from scratch.The methods mentioned in this Literature Review are summarised in Table 1.…”

Section: B Domain-specific Language Modelsmentioning

confidence: 99%

SpaceTransformers: Language Modeling for Space Systems

Berquand¹,

Darm

Riccardi

2021

IEEE Access

View full text Add to dashboard Cite

The transformers architecture and transfer learning have radically modified the Natural Language Processing (NLP) landscape, enabling new applications in fields where open source labelled datasets are scarce. Space systems engineering is a field with limited access to large labelled corpora and a need for enhanced knowledge reuse of accumulated design data. Transformers models such as the Bidirectional Encoder Representations from Transformers (BERT) and the Robustly Optimised BERT Pretraining Approach (RoBERTa) are however trained on general corpora. To answer the need for domainspecific contextualised word embedding in the space field, we propose SpaceTransformers, a novel family of three models, SpaceBERT, SpaceRoBERTa and SpaceSciBERT, respectively further pre-trained from BERT, RoBERTa and SciBERT on our domain-specific corpus. We collect and label a new dataset of space systems concepts based on space standards. We fine-tune and compare our domain-specific models to their general counterparts on a domain-specific Concept Recognition (CR) task. Our study rightly demonstrates that the models further pre-trained on a space corpus outperform their respective baseline models in the Concept Recognition task, with SpaceRoBERTa achieving significant higher ranking overall.

show abstract

LEGAL-BERT: The Muppets straight out of Law School

Cited by 317 publications

References 19 publications

What About the Precedent: An Information-Theoretic Analysis of Common Law

What About the Precedent: An Information-Theoretic Analysis of Common Law

NLP for Consumer Protection: Battling Illegal Clauses in German Terms and Conditions in Online Shopping

SpaceTransformers: Language Modeling for Space Systems

Contact Info

Product

Resources

About