In this paper, we integrate the spatial-spectral information of HSI samples into non-negative matrix factorization (NMF) for affinity matrix learning to address the issue of HSI clustering. This technique consists of three main components: i) oversegmentation for computing the spectral-spatial affinity matrix, ii) NMF with the guidance of the obtained affinity matrix and iii) density-based spectral clustering on the final affinity matrix. First, the HSI is oversegmented into superpixels via the entropy rate superpixel (ERS) algorithm. The spectral-spatial affinity matrix is defined based on the class-consistency assumption of all the HSI samples in each superpixel and the similar HSI samples between adjacent superpixels. Second, to integrate the spectral-spatial information into NMF, the obtained affinity matrix is used to guide the iterative process of NMF. The spectralspatial affinity matrix is then weighted by the affinity matrix in the obtained low-dimensional subspace to form the final affinity matrix. Third, density-based spectral clustering is applied to the final affinity matrix to obtain clustering maps. Experimental results on three public benchmark HSIs demonstrate that the proposed method is superior to the considered state-of-the-art baseline methods on both the computational cost and clustering accuracy.