Energy-constrained discriminant analysis

Abstract

Dimensionality reduction algorithms have become an indispensable tool for working with high-dimensional data in classification. Linear discriminant analysis (LDA) is a popular analysis technique used to project high-dimensional data into a lower-dimensional space while maximizing class separability. Although this technique is widely used in many applications, it suffers from overfitting when the number of training examples is on the same order as the dimension of the original data space. When overfitting occurs, the direction of the LDA solution can be dominated by low-energy noise and therefore the solution becomes non-robust to unseen data. In this paper, we propose a novel algorithm, energy-constrained discriminant analysis (ECDA), that overcomes the limitations of LDA by finding lower dimensional projections that maximize inter-class separability, while also preserving signal energy. Our results show that the proposed technique results in higher classification rates when compared to comparable methods. The results are given in terms of SAR image classification, however the algorithm is broadly applicable and can be generalized to any classification problem.

Muscle synergies in neuroscience and robotics: from input-space to task-space perspectives

“…Unlike the dimensionality reduction methods used so far, this approach would rely on supervised learning techniques to exploit information about the task. Possible alternatives to standard extraction algorithms include energy constrained discriminant analysis (Philips et al, 2009), the discriminant NMF (Buciu and Pitas, 2004), and the hybrid discriminant analysis (Yu et al, 2007). …”