Des-q: a Quantum Algorithm to Construct and Efficiently Retrain Decision Trees for Regression and Binary Classification

Abstract

Decision trees are widely used in machine learning for their simplicity in construction and interpretability. However, as data sizes grow, traditional methods for constructing and retraining decision trees become increasingly slow, scaling polynomially with the number of training examples. We introduce a novel quantum algorithm, Des-q, for constructing and retraining decision trees in regression and binary classification. Assuming the data stream produces small increments of new training examples, Des-q significantly reduces the tree retraining time, achieving poly-logarithmic time complexity in the number of examples, even accounting for the time needed to load the new examples into quantum-accessible memory. Our approach involves building a decision tree algorithm to perform k-piecewise linear tree splits at each internal node. These splits simultaneously generate multiple hyperplanes, dividing the feature space into k distinct regions. To determine the k suitable anchor points for these splits, we develop an efficient quantum-supervised clustering method, building upon the q-means algorithm by Kerenidis et al. Des-q first efficiently estimates each feature weight using a novel quantum technique to estimate the Pearson correlation. Subsequently, we employ weighted distance estimation to cluster the training examples in k disjoint regions based on nearest-neighbor matching and then proceed to expand the tree using the same procedure. We benchmark the performance of the simulated version of our algorithm against the state-of-the-art classical decision tree on multiple data sets with numerical features. Further, we showcase that the proposed algorithm exhibits similar performance to the state-of-the-art decision tree while significantly speeding up the periodic tree retraining.