Roweis Discriminant Analysis: A Generalized Subspace Learning Method
Benyamin Ghojogh,
Fakhri Karray,
Mark Crowley
Abstract:We present a new method which generalizes subspace learning based on eigenvalue and generalized eigenvalue problems. This method, Roweis Discriminant Analysis (RDA), is named after Sam Roweis to whom the field of subspace learning owes significantly. RDA is a family of infinite number of algorithms where Principal Component Analysis (PCA), Supervised PCA (SPCA), and Fisher Discriminant Analysis (FDA) are special cases. One of the extreme special cases, which we name Double Supervised Discriminant Analysis (DSD… Show more
“…Sammon mapping (Sammon, 1969) is a special case of metric MDS; hence, it is a nonlinear method. It is probably correct to call this method the first proposed nonlinear method for manifold learning (Ghojogh et al, 2019b). This method has different names in the literature such as Sammon's nonlinear mapping, Sammon mapping, and Nonlinear Mapping (NLM) (Lee & Verleysen, 2007).…”
Section: Sammon Mappingmentioning
confidence: 99%
“…In later approaches, Sammon mapping (Sammon, 1969) was proposed which is a special case of the distance-based metric MDS. One can consider Sammon mapping as the first proposed nonlinear manifold learning method (Ghojogh et al, 2019b). The disadvantage of Sammon mapping is its iterative solution of optimization, which makes this method a little slow.…”
“…Sammon mapping (Sammon, 1969) is a special case of metric MDS; hence, it is a nonlinear method. It is probably correct to call this method the first proposed nonlinear method for manifold learning (Ghojogh et al, 2019b). This method has different names in the literature such as Sammon's nonlinear mapping, Sammon mapping, and Nonlinear Mapping (NLM) (Lee & Verleysen, 2007).…”
Section: Sammon Mappingmentioning
confidence: 99%
“…In later approaches, Sammon mapping (Sammon, 1969) was proposed which is a special case of the distance-based metric MDS. One can consider Sammon mapping as the first proposed nonlinear manifold learning method (Ghojogh et al, 2019b). The disadvantage of Sammon mapping is its iterative solution of optimization, which makes this method a little slow.…”
“…1). For example, the non-zero pairs (2,20), (4,14), (13,6), (19,20), (17,6) in AW-FDA and the pairs (2,20), (4,14), (19,20), (17,14) in AW-KFDA make sense visually because of having glasses so their classes are close to one another.…”
Section: Comparison Of the Weightsmentioning
confidence: 99%
“…Treating closer classes need more attention because classifiers may more easily confuse them whereas classes far from each other are generally easier to separate. The same problem exists in Kernel FDA (KFDA) [3] and in most of subspace learning methods that are based on generalized eigenvalue problem such as FDA and KFDA [4]; hence, a weighting procedure might be more appropriate.…”
Fisher Discriminant Analysis (FDA) is a subspace learning method which minimizes and maximizes the intra-and inter-class scatters of data, respectively. Although, in FDA, all the pairs of classes are treated the same way, some classes are closer than the others. Weighted FDA assigns weights to the pairs of classes to address this shortcoming of FDA. In this paper, we propose a cosine-weighted FDA as well as an automatically weighted FDA in which weights are found automatically. We also propose a weighted FDA in the feature space to establish a weighted kernel FDA for both existing and newly proposed weights. Our experiments on the ORL face recognition dataset show the effectiveness of the proposed weighting schemes.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.