Structured AutoEncoders for Subspace Clustering

Peng, Xi; Feng, Jiashi; Xiao, Shijie; Yau, Wei-Yun; Zhou, Joey Tianyi; Yang, Songfan

doi:10.1109/tip.2018.2848470

Cited by 323 publications

(136 citation statements)

References 39 publications

Supporting

Mentioning

135

Contrasting

Order By: Relevance

“…Several works [26,25,4] proposed a type of the methodology that representations learned by auto-encoder were forced to follow a specific conventional prior structure related with self-expression, e.g., Sparse Subspace Clustering (SSC) [7] and Low-rank Representation (LRR) [16]. [15] proposed deep-encoder based row space recovery methodology to make conventional low-rank subspace clustering scalable and fast.…”

Section: Related Workmentioning

confidence: 99%

Deep Closed-Form Subspace Clustering

Seo¹,

Koo²,

Jeon³

2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

We propose Deep Closed-Form Subspace Clustering (DCFSC), a new embarrassingly simple model for subspace clustering with learning non-linear mapping. Compared with the previous deep subspace clustering (DSC) techniques, our DCFSC does not have any parameters at all for the self-expressive layer. Instead, DCFSC utilizes the implicit data-driven self-expressive layer derived from closed-form shallow auto-encoder. Moreover, DCFSC also has no complicated optimization scheme, unlike the other subspace clustering methods. With its extreme simplicity, DCFSC has significant memory-related benefits over the existing DSC method, especially on the large dataset. Several experiments showed that our DCFSC model had enough potential to be a new reference model for subspace clustering on large-scale high-dimensional dataset.

show abstract

Section: Related Workmentioning

confidence: 99%

Deep Closed-Form Subspace Clustering

Seo¹,

Koo²,

Jeon³

2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

show abstract

“…It provides an explicit non-linear mapping for the complex input data that is well-adapted to the subspace clustering, which yields significant improvement over the state-of-the-art subspace clustering solutions. A structured autoencoder in [34] introduces a global structure prior into the non-linear mapping. These deep subspace approaches mainly focus on the clustering or recognition problems, in which the network weights are well trained to exploit the similar information among the input data, instead of the differential information used for change detection task.…”

Section: Deep Subspace Clusteringmentioning

confidence: 99%

Differentially Deep Subspace Representation for Unsupervised Change Detection of SAR Images

Luo

et al. 2019

Remote Sensing

View full text Add to dashboard Cite

Temporal analysis of synthetic aperture radar (SAR) time series is a basic and significant issue in the remote sensing field. Change detection as well as other interpretation tasks of SAR images always involves non-linear/non-convex problems. Complex (non-linear) change criteria or models have thus been proposed for SAR images, instead of direct difference (e.g., change vector analysis) with/without linear transform (e.g., Principal Component Analysis, Slow Feature Analysis) used in optical image change detection. In this paper, inspired by the powerful deep learning techniques, we present a deep autoencoder (AE) based non-linear subspace representation for unsupervised change detection with multi-temporal SAR images. The proposed architecture is built upon an autoencoder-like (AE-like) network, which non-linearly maps the input SAR data into a latent space. Unlike normal AE networks, a self-expressive layer performing like principal component analysis (PCA) is added between the encoder and the decoder, which further transforms the mapped SAR data to mutually orthogonal subspaces. To make the proposed architecture more efficient at change detection tasks, the parameters are trained to minimize the representation difference of unchanged pixels in the deep subspace. Thus, the proposed architecture is namely the Differentially Deep Subspace Representation (DDSR) network for multi-temporal SAR images change detection. Experimental results on real datasets validate the effectiveness and superiority of the proposed architecture.For change detection using remotely sensed optical images, the most widely used criterion is difference operator [1] (for single channel images) or change vector analysis [6-8] (for multi-band/spectral images). Due to the temporal spectral variance caused by different atmospheric conditions, illumination and sensor calibration, image transformation has been widely used to yield robust change detection criteria. The core idea of the image transformation is to transform the multi-band/spectral image into a specific feature space, in which the unchanged temporal pixel pairs have similar representations while the changed ones differ from each other. Principal component analysis (PCA) [9][10][11] is one of the state-of-the-art operators for modeling temporal spectral difference of unchanged pixels. Beyond PCA, Kauth-Thomas transformation [12], Gram-Schmidt orthonormalization process [13,14], multivariate alteration detection [15,16] and slow feature analysis [17,18] theories have been used for optical image change detection. However, these algorithms are mainly designed for optical images and usually fail to deal with SAR images with speckle.Given SAR images, we may meet a more complex situation in which the multi-temporal images are in different feature spaces and changed/unchanged pixels are linearly non-sparable, due to the coherent imaging mechanism. Two main approaches have been developed in the literature: coherence change detection and incoherent change detection. The former uses the phase inf...

show abstract

“…As the popularity of neural networks in recent years [6], [9], [35], [47], we equip the LtDaHP scheme with an FNN-instance to show its outperformance. Taking inner weights as minimal Riesz energy points on a sphere and thresholds as equally spaced points (ESPs) in an interval, we can define an FNNrealization of LtDaHP.…”

Section: Introductionmentioning

confidence: 99%

Learning Through Deterministic Assignment of Hidden Parameters

Fang

Lin

Zhao

2020

IEEE Trans. Cybern.

View full text Add to dashboard Cite

Supervised learning frequently boils down to determining hidden and bright parameters in a parameterized hypothesis space based on finite input-output samples. The hidden parameters determine the nonlinear mechanism of an estimator, while the bright parameters characterize the linear mechanism. In traditional learning paradigm, hidden and bright parameters are not distinguished and trained simultaneously in one learning process. Such an one-stage learning (OSL) brings a benefit of theoretical analysis but suffers from the high computational burden. In this paper, we propose a twostage learning (TSL) scheme, learning through deterministic assignment of hidden parameters (LtDaHP), where we suggest to deterministically generate the hidden parameters by using minimal Riesz energy points on a sphere and equally spaced points in an interval. We theoretically show that with such deterministic assignment of hidden parameters, LtDaHP with a neural network realization almost shares the same generalization performance with that of OSL. Then, LtDaHP provides an effective way to overcome the high computational burden of OSL. We present a series of simulations and application examples to support the outperformance of LtDaHP.Index Terms-Supervised learning, neural networks, hidden parameters, bright parameters, learning rate.J. Fang and Z. Xu are with the School of Mathematics and Statistics, Xi'an Jiaotong University, Xi'an 710048, China. S. Lin is with the

show abstract

Structured AutoEncoders for Subspace Clustering

Cited by 323 publications

References 39 publications

Deep Closed-Form Subspace Clustering

Deep Closed-Form Subspace Clustering

Differentially Deep Subspace Representation for Unsupervised Change Detection of SAR Images

Learning Through Deterministic Assignment of Hidden Parameters

Contact Info

Product

Resources

About