Density estimation of high dimensional data using ICA and Bayesian networks

Abstract

This paper proposes a semi-non parametric density estimation framework for high-dimensional data. Dimensionality reduction is achieved by reorganizing the domain variables set into a junction tree of cliques each containing a small number of variables where factorization of the joint density into a tree is carried out by learning the Bayesian Network (BN) structure graph and by searching the maximum spanning tree over the moralized-triangulated graph of the obtained BN. To estimate the density of the junction tree elements, we propose a novel technique using local Independent Component Analysis (ICA) method based on fuzzy clustering. The main contribution relates to the development of a generic framework through a combination of three complimentary modules: (1) BN structure learning, (2) fuzzy clustering, and (3) linear ICA method. This allows us to exploit the separation power of recently developed ICA tools. Hence, depending on the data characteristics, the user can choose among a wide range of ICA and BN tools the most suitable one. We experimentally evaluated our approach in a supervised classification problem and the obtained results indicate an improvement in accuracy.