Copying Machine Learning Classifiers

Abstract

We study copying of machine learning classifiers, an agnostic technique to replicate the decision behavior of any classifier. We develop the theory behind the problem of copying, highlighting its properties, and propose a framework to copy the decision behavior of any classifier using no prior knowledge of its parameters or training data distribution. We validate this framework through extensive experiments using data from a series of well-known problems. To further validate this concept, we use three different use cases where desiderata such as interpretability, fairness or productivization constrains need to be addressed. Results show that copies can be exploited to enhance existing solutions and improve them adding new features and characteristics.

Differential Replication for Credit Scoring in Regulated Environments

“…We evaluate copies using copy accuracy . This value corresponds to the accuracy of the copy in the original test data [ 15 ]. For validation purposes, we also generate an additional test set composed of synthetic samples and report metrics on this set.…”

“…In what follows, we provide a brief overview of the copying process. A formal treatment of this process, as well as a display of its applications can be found in [ 15 ].…”

“…This optimization problem is always separable and, given enough capacity, zero training error is always achievable without hindering the generalization performance of the copy. In this context, we argue that ad hoc techniques can be used to better exploit these properties when building a copy [ 15 ]. In particular, assuming a sufficiently large set of synthetic data points is obtained, we can train the copies with no regard for overfitting.…”

“…Copying involves the joint optimization of the synthetic data points and the parameters of the copying model through Equation ( 3 ). This joint optimization is a complex process that requires the copy hypothesis space [ 15 ] to display certain features. In the simplest case, we can approximate this problem through a single iteration of an alternating optimization scheme by splitting the dual optimization into two independent sub-problems.…”

“…Depending on the considered level of knowledge, there exist different inheritance mechanisms that implement differential replication in practice. In this paper, we are interested in using inheritance by copying [ 15 ]. In particular, we discuss differential replication through copying in the context of non-client credit risk scoring.…”