Large Margin Subspace Learning for feature selection

Liu, Bo; Fang, Bin; Liu, Xinwang; Chen, Jie; Huang, Zhong; He, Xiping

doi:10.1016/j.patcog.2013.02.012

Cited by 26 publications

(8 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A similar thought of the large margin feature selection is shared by LMSL [14], which optimizes the criterion (17) to find an appropriate projection matrix W. We can indicate that LMSL may be seen as a special case of our proposed trace ratio framework. That is, the basic idea of our algorithm is to solve the reduced nonlinear eigenvalue problem (26), which has the same trace difference function as LMSL.…”

Section: Discussionmentioning

confidence: 77%

“…On the one hand, the above objective function is nonconvex since − n i=1 (A i − B i ) is an indefinite matrix, it is difficult to directly solve. Though there are several algorithms proposed to deal with this problem [14], [18], they achieve a suboptimal solution. On the other hand, as the sum of distances between x i and NM (x i ) is usually much larger than it between x i and NH (x i ), it may be better to assign a weight factor η to Tr W T XA i X T W for balance, and η should be smaller than 1 intuitively.…”

Section: Large Margin Feature Selection Based On Subspace Learningmentioning

confidence: 99%

“…The core idea of these methods is to use a transformation matrix to guide the feature selection according to the norm of its row/column vectors. Liu et al extended the Relief algorithm into the subspace learning and proposed a Large Margin Subspace Learning (LMSL) feature selection method [14]. This method defined a metric function to measure the margin of a given sample in the subspace.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Trace Ratio Criterion Based Large Margin Subspace Learning for Feature Selection

Luo

Han

2019

IEEE Access

View full text Add to dashboard Cite

In this paper, we propose a novel feature selection model based on subspace learning with the use of a large margin principle. First, we present a new margin metric described by a given instance and its nearest missing and nearest hit, which can be explained as the nearest neighbor with a different label and the same label, respectively. Specifically, for a given instance, the margin is the ratio of the distance of the nearest missing to that of the nearest hit rather than the difference of distances, which contributes to better balance since the distance to the nearest missing is usually much larger than the nearest hit. The proposed model seeks a subspace in which the margin metric is maximized. Moreover, considering that the nearest neighbors of a given sample are uncertain in the presence of many irrelevant features, we treat them as hidden variables and estimate the expectation of margin. To perform the feature selection, an 2,1-norm is imposed on the subspace projection matrix to enforce row sparsity. The resulting trace ratio optimization problem, which can be connected to a nonlinear eigenvalue problem, is hard to solve. Thus, we design an efficient iterative algorithm and present a theoretical analysis of the convergence. Finally, we evaluate the proposed method by comparing it against several other state-of-the-art methods. The extensive experiments on real-world datasets show the superiority of our proposed approach. INDEX TERMS Feature selection, relief, subspace learning, trace ratio.

show abstract

Section: Discussionmentioning

confidence: 77%

Section: Large Margin Feature Selection Based On Subspace Learningmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Trace Ratio Criterion Based Large Margin Subspace Learning for Feature Selection

Luo

Han

2019

IEEE Access

View full text Add to dashboard Cite

show abstract

“…In [63] feature selection is realized by differential evolution to reduce the search space. Some other feature selection algorithms are based on a maximum margin criteria [64], [65], [66], [67], [68], [69]. These methods are sample-based where the "margin" is defined as the difference between distance to the nearest same class sample (near-hit) and the nearest sample from opposite classes (near-miss).…”

Section: Feature Selectionmentioning

confidence: 99%

Feature Selection for Nonstationary Data: Application to Human Recognition Using Medical Biometrics

Komeili

Louis

Armanfard

et al. 2018

IEEE Trans. Cybern.

View full text Add to dashboard Cite

Electrocardiogram (ECG) and transient evoked otoacoustic emission (TEOAE) are among the physiological signals that have attracted significant interest in biometric community due to their inherent robustness to replay and falsification attacks. However, they are time-dependent signals and this makes them hard to deal with in across-session human recognition scenario where only one session is available for enrollment. This paper presents a novel feature selection method to address this issue. It is based on an auxiliary dataset with multiple sessions where it selects a subset of features that are more persistent across different sessions. It uses local information in terms of sample margins while enforcing an across-session measure. This makes it a perfect fit for aforementioned biometric recognition problem. Comprehensive experiments on ECG and TEOAE variability due to time lapse and body posture are done. Performance of the proposed method is compared against seven state-of-the-art feature selection algorithms as well as another six approaches in the area of ECG and TEOAE biometric recognition. Experimental results demonstrate that the proposed method performs noticeably better than other algorithms.

show abstract

“…Thus it is better to consider the class label to conduct the sparse coding and codebook learning, so that the generated sparse code and codebook can also benefit the classification problem. For example, in [17], the codebook is learned by using the class labels to maximize the margins of samples [55]- [59].…”

Section: Introductionmentioning

confidence: 99%

Ideal code constrained supervised sparse coding

Liao¹,

Williams²

2014

JCP

View full text Add to dashboard Cite

In this paper, we proposed a novel sparse coding algorithm by using the class labels to constrain the learning of codebook and sparse code. We not only use the class label to train the classifier, but also use it to construct class conditional codewords to make the sparse code as discriminative as possible. We first construct ideal sparse codes with regarding to the class conditional codewords, and then constrain the learned sparse codes to the ideal sparse codes. We proposed a novel loss function composed of parse reconstruction error, classification error, and the ideal sparse code constrain error. This problem can be optimized by using the transitional KSVD method. In this way, we may learn a discriminative classifier and a discriminative codebook simultaneously. Moreover, using this codebook, the learnt the sparse codes of the same class are similar to each other. Finally, exhaustive experimental results show that the proposed algorithm outperforms other sparse coding methods.

show abstract

Large Margin Subspace Learning for feature selection

Cited by 26 publications

References 25 publications

Trace Ratio Criterion Based Large Margin Subspace Learning for Feature Selection

Trace Ratio Criterion Based Large Margin Subspace Learning for Feature Selection

Feature Selection for Nonstationary Data: Application to Human Recognition Using Medical Biometrics

Ideal code constrained supervised sparse coding

Contact Info

Product

Resources

About