Generating Multi-Codebook Neural Network by Using Intelligent Gaussian Mixture Model Clustering Based on Histogram Information for Multi-Modal Data Classification

Abstract

One of the open challenges in machine learning is multi-modal data classification. A classifier model needs to be enhanced to deal with multi-modal data. This study is proposed to develop multi-codebook neural networks using intelligent Gaussian mixture model clustering for multi-modal data classification. The intelligent Gaussian mixture model clustering is developed in this study prior to the development of multicodebook models. The method analyzes the gradient of input data histogram to find the number of generated mixtures and cluster the data. The proposed multi-codebook neural network model has three variants based on the rules to fund the number of clusters. The experiment result showed that the proposed three variants of multi-codebook models performed well in the synthetic and benchmark datasets. The proposed model improved the original method by 24.14%, 15.97%, and 3.71% accuracy and 0.3510, 0.0487, and 0.2031 kappa for synthetic datasets, benchmark datasets, and overall datasets respectively. By using the ANOVA test, we have proved that all three variants of the proposed multi-codebook neural network were proved to have significant improvements compared to the original version.INDEX TERMS multi-modal, classification, multi-codebook, neural networks, intelligent clustering, Gaussian mixture model, histogram.