Deep Least Squares Fisher Discriminant Analysis

Díaz-Vico, David; Dorronsoro, José R.

doi:10.1109/tnnls.2019.2906302

Cited by 27 publications

(16 citation statements)

References 38 publications

(31 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Second, in this study we only focus on binary classification, it is also necessary to further explore the algorithms to solve imbalanced multiclass classification tasks. To achieve such goal, we will refer to the definition of the total AUC for multiclass classification in [58] and modify the relevant objective functions accordingly. Third, it is worthwhile to extend the direct AUC optimization to different algorithms in the family of kernel ridge regression to further improve the classification performances.…”

Section: Discussionmentioning

confidence: 99%

AUC-Based Extreme Learning Machines for Supervised and Semi-Supervised Imbalanced Classification

Wang

Wong

2021

IEEE Trans. Syst. Man Cybern, Syst.

View full text Add to dashboard Cite

Extreme learning machines (ELM) has been theoretically and experimentally proved to achieve promising performance at a fast learning speed for supervised classification tasks. However, it does not perform well on imbalanced binary classification tasks and tends to get biased towards the majority class. Besides, since a large amount of training data with labels are not always available in the real world, there is an urgent demand to develop an efficient semi-supervised version of ELM for imbalanced binary classification tasks. In this paper, owing to the distinct insensitivity of Area Under the ROC curve (AUC) to both class skews and changes of class distributions, we focus the study on integrating AUC maximization into the ELM framework to tackle with imbalanced binary classification tasks well. By demystifying the AUC metric with the ELM framework, we develop a new AUC based ELM called AUC-ELM for imbalanced binary classification, which essentially is revealed to be equivalent to an ELM on another transformed data space. Accordingly, its semi-supervised version called SAUC-ELM is also developed. Both AUC-ELM and SAUC-ELM have the distinctive merits:(1) they share the advantage of ELM in both generalization capability and training efficiency, and further uniquely tailored for imbalanced binary classification tasks; (2) In contrast to the existing imbalanced variants of ELM such as Class-specific Cost Regulation ELM and Semi-supervised ELM, they have fewer parameters to tune, thereby reducing the computational cost for model selection. Experiments on a heap of datasets show that both AUC-ELM and SAUC-ELM outperform the other comparative methods in terms of both classification performance and training speed.

show abstract

Section: Discussionmentioning

confidence: 99%

AUC-Based Extreme Learning Machines for Supervised and Semi-Supervised Imbalanced Classification

Wang

Wong

2021

IEEE Trans. Syst. Man Cybern, Syst.

View full text Add to dashboard Cite

show abstract

“…Deep discriminant analysis metric learning methods use the idea of Fisher discriminant analysis (Fisher, 1936;Ghojogh et al, 2019b) in deep learning, for learning an embedding space which separates classes. Some of these methods are deep probabilistic discriminant analysis (Li et al, 2019), discriminant analysis with virtual samples (Kim & Song, 2021), Fisher Siamese losses (Ghojogh et al, 2020f), and deep Fisher discriminant analysis (Díaz-Vico et al, 2017;Díaz-Vico & Dorronsoro, 2019). The Fisher Siamese losses were already introduced in Section 5.3.13.…”

Section: Deep Discriminant Analysis Metric Learningmentioning

confidence: 99%

“…F is the Frobenius norm, X ∈ R n×d is the rowwise stack of data points, Y := HEΠ −(1/2) ∈ R n×c where H := I − (1/n)11 ∈ R n×n is the centering matrix, E ∈ {0, 1} n×c is the one-hot-encoded labels stacked row-wise, Π ∈ R c×c is the diagonal matrix whose (l, l)-th element is the cardinality of the l-th class. Deep Fisher discriminant analysis (Díaz-Vico et al, 2017;Díaz-Vico & Dorronsoro, 2019) implements Eq. ( 183) by a nonlinear neural network with loss function:…”

Section: Deep Fisher Discriminant Analysismentioning

confidence: 99%

Spectral, Probabilistic, and Deep Metric Learning: Tutorial and Survey

Ghojogh¹,

Ghodsi²,

Karray³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Suppose µ (r) ∈ R d , c, n, and U ∈ R d×d denote the mean of r-th class, the number of classes, the total sample size, and the projection matrix in FDA, respectively. Although some methods solve FDA using least squares problem [5,6], the regular FDA [2] maximizes the Fisher criterion [7]:…”

Section: Fisher and Kernel Discriminant Analysismentioning

confidence: 99%

Weighted Fisher Discriminant Analysis in the Input and Feature Spaces

Ghojogh,

Sikaroudi,

Tizhoosh

et al. 2020

Preprint

View full text Add to dashboard Cite

Fisher Discriminant Analysis (FDA) is a subspace learning method which minimizes and maximizes the intra-and inter-class scatters of data, respectively. Although, in FDA, all the pairs of classes are treated the same way, some classes are closer than the others. Weighted FDA assigns weights to the pairs of classes to address this shortcoming of FDA. In this paper, we propose a cosine-weighted FDA as well as an automatically weighted FDA in which weights are found automatically. We also propose a weighted FDA in the feature space to establish a weighted kernel FDA for both existing and newly proposed weights. Our experiments on the ORL face recognition dataset show the effectiveness of the proposed weighting schemes.

show abstract

Deep Least Squares Fisher Discriminant Analysis

Abstract: https://repositorio.uam.es Esta es la versión de autor del artículo publicado en: This is an author produced version of a paper published in:

Cited by 27 publications

References 38 publications

AUC-Based Extreme Learning Machines for Supervised and Semi-Supervised Imbalanced Classification

AUC-Based Extreme Learning Machines for Supervised and Semi-Supervised Imbalanced Classification

Spectral, Probabilistic, and Deep Metric Learning: Tutorial and Survey

Weighted Fisher Discriminant Analysis in the Input and Feature Spaces

Contact Info

Product

Resources

About