Minimum Viable Model Estimates for Machine Learning Projects

Abstract

Prioritization of machine learning projects requires estimates of both the potential ROI of the business case and the technical difficulty of building a model with the required characteristics. In this work we present a technique for estimating the minimum required performance characteristics of a predictive model given a set of information about how it will be used. This technique will result in robust, objective comparisons between potential projects. The resulting estimates will allow data scientists and managers to evaluate whether a proposed machine learning project is likely to succeed before any modelling needs to be done. The technique has been implemented into the open source application MinViME (Minimum Viable Model Estimator) which can be installed via the PyPI python package management system, or downloaded directly from the GitHub repository. Available at https://github.com/john-hawkins/MinViME.

MinViME/Minimum Viable Model Estimator

“…The MinViME package has permitted us to run simulation experiments to answer questions about the relationship between the landscape of business problems and the minimum viable models that would solve them [7]. These simulations form the basis of a new set of heuristics for managers of data science and machine learning projects.…”

“…In our theoretical work we demonstrated that the MinViME system can produce a range of metrics from both analytical and numerical techniques [7]. These metrics include: minimal precision, estimated…”