BERT-PLI: Modeling Paragraph-Level Interactions for Legal Case Retrieval

Abstract

<jats:p>Legal case retrieval is a specialized IR task that involves retrieving supporting cases given a query case. Compared with traditional ad-hoc text retrieval, the legal case retrieval task is more challenging since the query case is much longer and more complex than common keyword queries. Besides that, the definition of relevance between a query case and a supporting case is beyond general topical relevance and it is therefore difficult to construct a large-scale case retrieval dataset, especially one with accurate relevance judgments. To address these challenges, we propose BERT-PLI, a novel model that utilizes BERT to capture the semantic relationships at the paragraph-level and then infers the relevance between two cases by aggregating paragraph-level interactions. We fine-tune the BERT model with a relatively small-scale case law entailment dataset to adapt it to the legal scenario and employ a cascade framework to reduce the computational cost. We conduct extensive experiments on the benchmark of the relevant case retrieval task in COLIEE 2019. Experimental results demonstrate that our proposed method outperforms existing solutions.</jats:p>

PRILJ: an efficient two-step method based on embedding and clustering for the identification of regularities in legal case judgments

“…The same line of research has been explored in the Competition On Legal Information Extraction/Entailment (COLIEE), during which several tasks related to the legal domain have been solved with the support of embedding techniques. Among the approaches related to the present paper, it is worth mentioning BERT-PLI (Shao et al 2020), that adopts BERT to capture the semantic relationships at the paragraph-level and then infers the relevance between two cases by aggregating paragraph-level interactions. Analogously to LEGAL-BERT, the BERT model in BERT-PLI is fine-tuned with a dataset related to the legal field.…”

“…• LEGAL-BERT-BASE, that is the LEGAL-BERT model 3 fine-tuned by Chalkidis et al (2020) using a wide set of legal documents related to EU, UK and US law; • LEGAL-BERT-SMALL, that is the LEGAL-BERT model 3 fine-tuned by Chalkidis et al (2020) using the same set of documents adopted for LEGAL-BERT-BASE, but in a lower-dimensional embedding space; • LEGAL-BERT-EURLEX, that is the LEGAL-BERT model 3 fine-tuned by Chalkidis et al (2020) using the EUR-LEX dataset; • BERT-PLI, that is the system BERT-PLI 4 based on BERT, fine-tuned with a small set of legal documents, proposed by Shao et al (2020) in the Competition On Legal Information Extraction/Entailment (COLIEE).…”

Searching for Legal Documents at Paragraph Level: Automating Label Generation and Use of an Extended Attention Mask for Boosting Neural Models of Semantic Similarity

“…While this trend is in its early stages, its maturation could help to deal with some of the above-mentioned limitations. However, it was observed by many authors (Alberts et al 2020, Bhattacharya et al 2020, Chalkidis et al, 2020, Draijer 2019, Raghav et al 2016, Shao et al 2020, Van Opijnen & Santos 2017, Xiao et al 2019, Wang et al 2019, Zhong et al 2020) that current neural systems for natural language understanding that perform very well in non-legal domains do not transfer easily to tasks in the legal domain, for a variety of reasons that make this domain especially challenging (see Table 1).…”

“…For a user it can be helpful to first find the most semantically similar paragraphs with a few relevant legal concepts in a lengthy and complex case document, before reading the whole document. This way of defining the task also has another practical advantage, in the sense that many state-of-the-art (neural) models would struggle to encode the semantics of a very long text with hundreds of sentences in a useful way, while text at the level of one or a few sentences will be a more realistic input text to such models (Alberts et al 2020, Shao et al 2020). In the best case, such semantic similarity models at paragraph level should be invariant to differences in the input that do not matter for approximating relevance in this task, while they will be more selective to differences that do matter (Neculoiu et al 2016).…”

“…The above problem can therefore be formulated as a specialized Information Retrieval task that involves searching through a corpus of legal documents, before ranking them according to their relevance. While the concept of relevance in this context is conceptually and practically complex, we will focus our attention on a key aspect, the semantic similarity between a query and a candidate hit (Shao et al 2020). Early attempts at developing such systems often involved the use of Boolean expressions as queries, where a document…”