Predicting judicial decisions of the European Court of Human Rights: a Natural Language Processing perspective

Abstract

Recent advances in Natural Language Processing and Machine Learning provide us with the tools to build predictive models that can be used to unveil patterns driving judicial decisions. This can be useful, for both lawyers and judges, as an assisting tool to rapidly identify cases and extract patterns which lead to certain decisions. This paper presents the first systematic study on predicting the outcome of cases tried by the European Court of Human Rights based solely on textual content. We formulate a binary classification task where the input of our classifiers is the textual content extracted from a case and the target output is the actual judgment as to whether there has been a violation of an article of the convention of human rights. Textual information is represented using contiguous word sequences, i.e., N-grams, and topics. Our models can predict the court's decisions with a strong accuracy (79% on average). Our empirical analysis indicates that the formal facts of a case are the most important predictive factor. This is consistent with the theory of legal realism suggesting that judicial decision-making is significantly affected by the stimulus of the facts. We also observe that the topical content of a case is another important feature in this classification task and explore this relationship further by conducting a qualitative analysis.

Measuring the judicial performance of the European Court of Human Rights

“…Will the next step consist in replacing judges by electronic predictive models for a certain number of cases (well-established and repetitive cases)? 59 Thus these quantitative criteria are clearly questionable. Former Registrar Erik Fribergh himself acknowledged that they cannot properly assess the extent to which the Court has fulfilled its mission of ensuring observance with the rights and freedoms set forth in the ECHR (which is its raison d'être according to Article 19), which depends on 'the nature of the cases' and the 'quality of the decision'.…”

HYPO’S legacy: introduction to the virtual special issue

“…The system produced a binary classification into violation and no violation. The task in Aletras et al (2016) is thus more like that of of SMILE?IBP (although SMILE was looking for individual factors rather than the outcome) which, because of the difficulty of the SMILE task compared with the IBP task, performs much closer to 70%. SMILE.…”

“…6 Where large quantities of cases are available, they tend to be subjected to machine learning techniques rather than manual analysis. A recent example is Aletras et al (2016), which will be discussed in more detail later (see footnote 12). suggestion is that the decision in the precedent should be applied to the current case (following the principle of stare decisis, which means let the decision stand).…”

“…The dataset has been subsequently reused in Johnston and Governatori (2003), Možina et al (2005), Wardeh et al (2009) and elsewhere. 12 Although just 79% was considered enough for the work reported in Aletras et al (2016) to make a stir in the national press and radio in the UK in October 2016: e.g The Guardian at www.theguardian.com/ technology/2016/oct/24/artificial-intelligence-judge-university-college-london-computer-scientists and the BBC, www.bbc.co.uk/news/technology-37727387 This perhaps a reflection of the upsurge in interest in applications of AI and Law in the UK. See for example the 2016 Law Society of England and Wales panel at www.youtube.com/watch?v=8jPB-4Y3jLg and a special edition of the BBC Radio 4 programme Law in Action at www.bbc.co.uk/programmes/b07dlxmj.…”

“…See for example the 2016 Law Society of England and Wales panel at www.youtube.com/watch?v=8jPB-4Y3jLg and a special edition of the BBC Radio 4 programme Law in Action at www.bbc.co.uk/programmes/b07dlxmj. The program in Aletras et al (2016) took the text of published judgements from the European Court of Human Rights (i.e material produced ex post) acting as proxies for the unavailable applications and briefs. As the authors observe, ex ante documents would be the input to an actual prediction system.…”