There are many problems of division in natural and social sciences, and with the development of science and technology, the requirement for division is also increasing. It is difficult to divide accurately by experience and expertise alone, and the most important research branch of the division problem is the clustering algorithm. It is to group similar samples into one class and divide the elements with large differences into different classes. Due to the simplicity and efficiency of the clustering algorithm, it is widely used in image segmentation. The conventional Spectral Clustering (SC) algorithm cannot recognize nonconvex data and has the disadvantage of strong dependence on biased parameter values. To address this problem, the Gravity-based Adaptive Spectral Clustering (GASC) algorithm is proposed in this study. Based on the conventional SC algorithm, the algorithm uses gravity to calculate the similarity (gravity) between data, and uses information entropy and adaptive enhancement (AdaBoost) algorithms to obtain the weights of correct cluster sampling points and wrong cluster sampling points in each cluster, so as to reduce the dependence of the algorithm on bias parameters and reduce the number of wrongly divided sample points. Meanwhile, the GASC algorithm is applied to image segmentation. The implementation process includes three stages: image preprocessing, feature extraction, and clustering. The comparison experiments show that the mean values of normalization and accuracy of the GASC algorithm are improved compared with other clustering algorithms, and the segmentation accuracy for images is higher.