Evaluating multi-label classifiers and recommender systems in the financial service sector

Abstract

The objective of this paper is to evaluate multi-label classification techniques and recommender systems for cross-sell purposes in the financial services sector. We carried out three analyses using data obtained from an international financial services provider. First, we tested four multi-label classification techniques, of which the two problem transformation methods were combined with several base classifiers. Second, we benchmarked the performance of five state-of-the-art recommender approaches. Third, we compared the best performing multi-label classification and recommender approaches with each other. The results identify user-based collaborative filtering as the top performing recommender system, with a cross-validated F1 measure of 42.20% and G-mean of 42.64%. Classifier chains binary relevance with adaboost and binary relevance with random forest are the top performing multilabel classification algorithms for respectively F1 measure and G-mean, yielding a crossvalidated F1 measure of 53.33% and G-mean of 54.37%. The statistical comparison between the best performing approaches confirms the superiority of multi-label classification techniques. Our study provides important recommendations for financial services providers, who are interested in the most effective methods to determine cross-sell opportunities. In previous studies, multi-label classification techniques and recommender systems were always investigated independently of each other. To the best of our knowledge, our study is therefore the first to compare both techniques in the financial services sector.

MLCM: Multi-Label Confusion Matrix

“…Bogaert et al [21] evaluated multi-label classification techniques and recommender systems for cross-sell purposes in the financial services sector. They used five overall evaluation metrics with more focus on harmonic and geometric mean of precision and recall in order to evaluate the performance of the analytical techniques.…”

Multi-label Ranking: Mining Multi-label and Label Ranking Data

“…Moreover, evaluation measures need to be fine-tuned to this setting of emerging labels [108,144]. https://cs.uni-paderborn.de/?id=63912 Second, a recent study [11] compared recommender system and multi-label classification techniques concluding that AdaBoost with CC chains and BR with multilabel random forest outperform the best recommender system methods in a given cross-selling setting. However, state-of-the-art multi-label deep learning methods and extreme multi-label methods should be able to do even better but have not been considered in the above study.…”

Improved Double‐Layer Structure Multilabel Classification Model via Optimal Sequence and Attention Mechanism

“…Usually, it is very challenging to extend the theory of single-label classifcation to multilabel classifcation. With the development of machine learning, multilabel classifcation algorithms can be applied to imaging, recommendation systems, medical diagnosis, information retrieval, and many other felds [1][2][3][4][5][6][7][8]. In recent years, an ocean of research works accepted by top conferences (e.g., ACL, AAAI, COLING, KDD, NIPS, ICDM, CIKM, INTERSPEECH, ICML, and IJCAI) proposed technologies and solutions for multilabel classifcation.…”