Dissecting Moneyball: Improving Classification Model Interpretability in Baseball Pitch Prediction

Abstract

Data science, where technical expertise meets domain knowledge, is collaborative by nature. Complex machine learning models have achieved human-level performance in many areas, yet they face adoption challenges in practice due to limited interpretability of model outputs, particularly for users who lack specialized technical knowledge. One key question is how to unpack complex classification models by enhancing their interpretability to facilitate collaboration in data science research and application. In this study, we extend two state-of-the-art methods for drawing finegrained explanations from the results of classification models. The main extensions include aggregating explanations from individual instances to a user-defined aggregation level, and providing explanations with the original features rather than engineered representations. We use the prediction of baseball pitch outcome as a case to evaluate our extended methods. The experiment results of the methods with real sensor data demonstrate their improved interpretability while preserving superior prediction performance.

Predicting baseball pitcher efficacy using physical pitch characteristics

“…to predict different aspects of baseball using sabermetrics. For instance, Lee et al and Hickey et al used ML models to predict a thrown pitch's outcome (2,3). Furthermore, another study by Bock used sabermetrics and ML models to predict pitchers' short-term and long-term efficacy on their particular teams (4).…”

Making sense of the black‐boxes: Toward interpretable text classification using deep learning models

“…Interpretable machine learning techniques can be characterized in three dimensions: model‐specificity (specific vs. agnostic), generalizability/scope (local vs. global) (Rai, 2020), and stage of data generation (ranging from data to results), as shown in Figure 1. Obtaining model‐specific global explanations (see upper‐left quadrant in Figure 1) is the most developed subarea in interpretable machine learning (Hickey et al, 2020). For instance, feature importance scoring is widely employed to understand an individual feature's contribution to the model.…”

“…To generate interpretations at a user-defined level (e.g., a few instances), one solution is to first enumerate the interpretations on all instances, and then aggregate them (Hickey et al, 2020). To improve the scalability of interpretation aggregation, we propose an approach that identifies the most indicative features in the following steps.…”