Improved customer choice predictions using ensemble methods

Abstract

In this paper various ensemble learning methods from machine learning and statistics are considered and applied to the customer choice modeling problem. The application of ensemble learning usually improves the prediction quality of flexible models like decision trees and thus leads to improved predictions. We give experimental results for two real-life marketing datasets using decision trees, ensemble versions of decision trees and the logistic regression model, which is a standard approach for this problem. The ensemble models are found to improve upon individual decision trees and outperform logistic regression.Next, an additive decomposition of the prediction error of a model is considered known as the bias/variance decomposition. A model with a high bias lacks the flexibility to fit the data well. A high variance indicates that a model is instable with respect to different datasets. Decision trees have a high variance component and a low bias component in the prediction error, whereas logistic regression has a high bias component and a low variance component. It is shown that ensemble methods aim at minimizing the variance component in the prediction error while leaving the bias component unaltered. Bias/variance decompositions for all models for both customer choice datasets are given to illustrate these concepts.

Data Mining Based Tax Audit Selection: A Case Study of a Pilot Project at the Minnesota Department of Revenue

“…Boosting Naïve Bayes was applied to claim fraud diagnosis [31]. Studies showed that MultiBoosting performed well in the prediction of customer choice [30], the prediction of financial distress [26], and the assessment of customer credit quality [21].…”

Local recurrence based performance prediction and prognostics in the nonlinear and nonstationary systems

“…Prediction of state and performance is critical to the assurance of integrity and sustainability of many complex natural and engineering physical systems [1][2][3][4]. In the modern manufacturing enterprises, timely and frequent prediction of manufacturing system Key Performance Indicators (KPI), including throughput rates, throughput losses due to breakdowns, blocking and starving, Work In Process (WIP) levels, etc., is essential to support day-today manufacturing planning and control decisions.…”

Random Subspace Learning (RASSEL) with data driven weighting schemes

“…This technique is known as ensemble learning (aggregation). In spite of the underlying algorithm used, the ensemble learning technique most of the time (on average) outperforms the single learning technique, especially for prediction purposes [21]. There are many approaches of performing ensemble learning.…”