Online streaming feature selection for multigranularity hierarchical classification learning

Abstract

Hierarchical classification learning is a hot research topic in machine learning and data mining domains, and many feature selection algorithms with category hierarchy have been proposed. However, existing algorithms assume that the feature space of data is completely obtained in advance, and ignore its uncertainty and dynamicity. To address these problems, we propose an online streaming feature selection framework with a hierarchical structure to solve the above two problems simultaneously. First, we apply the hierarchical relationship between nodes in a hierarchical structure to the Relief algorithm, so that it can be used to compute the weights of dynamic features. Second, we dynamically select important features for each internal node via comparing the magnitude of the weights of features on these nodes with their parent and sibling nodes. In addition, we perform redundancy analysis of features by calculating the covariance between features to obtain a superior online feature subset for each internal node. Finally, the proposed algorithm is compared with six online streaming feature selection methods on six hierarchical data sets, and experimental results shows that the classification performance of the proposed algorithm is effective.

Online streaming feature selection based on hierarchical structure information

“…Hierarchical data can generally be categorized into two primary types: tree‐based and graph‐based structures 41 . In this paper, our primary focus is a tree‐based format.…”

Online feature selection for hierarchical classification learning based on improved ReliefF

“…Definition (Reference 28). Matrix $$$ Y\subseteq {R}^{d\times d} $$$ is used to describe the sibling relationship between nodes, $$$ d $$$ is the number of leaf nodes in the hierarchical tree structure of classes, the sibling relationship of node $$$ {l}_i $$$ is represented by $$$ {Y}_i $$$, and the sibling relationship between nodes $$$ {l}_i $$$ and $$$ {l}_j $$$ is represented by $$$ {Y}_{ij} $$$.…”

“…At present, many scholars have conducted in-depth research on the flow feature selection. [25][26][27][28][29] Zhou et al 25 defined an adaptive density neighborhood relation without prior information based on neighborhood rough set, and proposed a new online streaming feature selection algorithm. Li et al 26 proposed a causality-based online streaming feature selection algorithm with neighborhood conditional mutual information.…”

“…Transforming the feature selection problem under static and fixed conditions into an online streaming feature selection problem under the concept of flow features plays a key role in dealing with such practical problems. At present, many scholars have conducted in‐depth research on the flow feature selection 25‐29 . Zhou et al 25 defined an adaptive density neighborhood relation without prior information based on neighborhood rough set, and proposed a new online streaming feature selection algorithm.…”

“…Bai et al 27 constructed a new hierarchical data‐oriented kernelized fuzzy rough model with sibling strategy, and proposed an online streaming feature selection framework for large‐scale hierarchical classification task. Wang et al 28 applied the hierarchical relationship between nodes in a tree structure to the Relief algorithm, but this algorithm required to specify the relevant parameters. Wang et al 29 proposed an online streaming feature selection algorithm for hierarchical classification based on the adaptive ReliefF algorithm, this algorithm did not need to specify parameters in advance, but did not consider the interaction between features.…”