Predicting the Behavior of the Supreme Court of the United States: A General Approach

Abstract

Building upon developments in theoretical and applied machine learning, as well as the efforts of various scholars including Guimerà and Sales-Pardo (2011), Ruger et al. (2004), andMartin et al. (2004), we construct a model designed to predict the voting behavior of the Supreme Court of the United States. Using the extremely randomized tree method first proposed in Geurts et al. (2006), a method similar to the random forest approach developed in Breiman (2001), as well as novel feature engineering, we predict more than sixty years of decisions by the Supreme Court of the United States . Using only data available prior to the date of decision, our model correctly identifies 69.7% of the Court's overall affirm / reverse decisions and correctly forecasts 70.9% of the votes of individual justices across 7,700 cases and more than 68,000 justice votes. Our performance is consistent with the general level of prediction offered by prior scholars. However, our model is distinctive as it is the first robust, generalized, and fully predictive model of Supreme Court voting behavior offered to date. Our model predicts six decades of behavior of thirty Justices appointed by thirteen Presidents. With a more sound methodological foundation, our results represent a major advance for the science of quantitative legal prediction and portend a range of other potential applications, such as those described in Katz (2013).

Using machine learning to predict decisions of the European Court of Human Rights

“…Most of these studies use manually collected and coded case law. Many studies use the Supreme Court Database, which contains manually collected and expertly-coded data on the US Supreme Court's behaviour of the last two hundred years (Katz et al 2017). A large amount of these studies analyse the relationship between gender or political background of judges and their decision-making (see Epstein et al 2013;Rachlinski and Wistrich 2017;Frankenreiter 2018).…”

“…Again, researchers in the United States were the first to use this technique to predict the courts' decisions or voting behaviour of judges (Katz 2012;Wongchaisuwat et al 2017). Recently, Katz et al (2017) developed a prediction model that aims to predict whether the US Supreme Court as a whole affirms or reverses the status quo judgement, and whether each individual Justice of the Supreme Court will vote to affirm or reverse the status quo judgement. Their model achieved an accuracy of 70.2% at the case outcome level and 71.9% at the justice vote level.…”

Scalable Methods for Annotating Legal-Decision Corpora

“…The accuracy of predictive legal models is highest, and explanatory capability greatest, when the prior decisions are represented in terms of features manually engineered to express exactly the most relevant aspects of the prior case (Katz et al, 2017). However, this approach is not scalable.…”

Proceedings of the Natural Legal Language Processing Workshop 2019

“…Predictive legal models have the potential to improve both the delivery of services to citizens and the efficiency of agency decision processes, e.g., by making benefits adjudications faster and more transparent, and by enabling decision-support tools for evaluating benefits claims. The accuracy of predictive legal models is highest, and explanatory capability greatest, when the prior decisions are represented in terms of features manually engineered to express exactly the most relevant aspects of the prior case (Katz et al, 2017). However, this approach is not scalable.…”