Adaptive local dissimilarity measures for discriminative dimension reduction of labeled data

Abstract

Due to the tremendous increase of electronic information with respect to the size of data sets as well as their dimension, dimension reduction and visualization of high-dimensional data has become one of the key problems of data mining. Since embedding in lower dimensions necessarily includes a loss of information, methods to explicitly control the information kept by a specific dimension reduction technique are highly desirable. The incorporation of supervised class information constitutes an important specific case. The aim is to preserve and potentially enhance the discrimination of classes in lower dimensions. In this contribution we use an extension of prototype-based local distance learning, which results in a nonlinear discriminative dissimilarity measure for a given labeled data manifold. The learned local distance measure can be used as basis for other unsupervised dimension reduction techniques, which take into account neighborhood information. We show the combination of different dimension reduction techniques with a discriminative similarity measure learned by an extension of Learning Vector Quantization (LVQ) and their behavior with different parameter settings. The methods are introduced and discussed in terms of artificial and real world data sets.

Limited Rank Matrix Learning, discriminative dimension reduction and visualization

“…The concept of LiRaM LVQ can also be expanded to the use of localized rectangular matrices, representing several local linear projections. The global combination of these local linear patches by means of charting is discussed in Brand (2003), and Bunte, Hammer, Wismüller, and Biehl (2010b).…”

“…These algorithms have been employed successfully in a variety of scientific and commercial applications, including image analysis, bioinformatics, robotics, etc. (Biehl, Ghosh, & Hammer, 2007;Bojer, Hammer, Schunk, & von Toschanowitz, 2001;Bunte, Biehl, Petkov, & Jonkman, 2009;Bunte, Hammer, Schneider, & Biehl, 2009;Bunte, Hammer, Wismüller, & Biehl, 2010a;Hammer, Strickert, & Villmann, 2005a;Hammer & Villmann, 2002;Villmann, Merenyi, & Hammer, 2003). The method is easy to implement and its complexity is controlled by the user in a straightforward way.…”

“…Kernel techniques extend these settings to nonlinear projections (Baudat & Anouar, 2000;Ma, Qu, & Wong, 2007). Adaptive dissimilarity measures which modify the metric used for projection according to the given auxiliary information have been introduced in Kaski, Sinkkonen, J, and Peltonen (2001), Peltonen, Klami, and Kaski (2004), and Bunte et al (2010a). The resulting metric can be integrated into various techniques such as SOM, MDS, or a recent information theoretic model for data visualization (Kaski et al, 2001;Peltonen et al, 2004;Venna, Peltonen, Nybo, Aidos, & Kaski, 2010).…”

“…Note that the dimension reduction itself is done in terms of a global linear projection. The concept of using local dissimilarities in combination with non-linear dimension reduction and visualization was recently discussed in Bunte et al (2010b).…”

Learning effective color features for content based image retrieval in dermatology

“…Detailed information about the algorithm, parameters, running time and complexity can be found, for example in [11,2,30,7].…”