Bilinear mixture model-based methods have recently shown promising capability in nonlinear spectral unmixing. However, relying on the endmembers extracted in advance, their unmixing accuracies decrease, especially when the data is highly mixed. In this paper, a strategy of geometric projection has been provided and combined with constrained nonnegative matrix factorization for unsupervised nonlinear spectral unmixing. According to the characteristics of bilinear mixture models, a set of facets are determined, each of which represents the partial nonlinearity neglecting one endmember. Then, pixels' barycentric coordinates with respect to every endmember are calculated in several newly constructed simplices using a distance measure. In this way, pixels can be projected into their approximate linear mixture components, which reduces greatly the impact of collinearity. Different from relevant nonlinear unmixing methods in the literature, this procedure effectively facilitates a more accurate estimation of endmembers and abundances in constrained nonnegative matrix factorization. The updated endmembers are further used to reconstruct the facets and get pixels' new projections. Finally, endmembers, abundances, and pixels' projections are updated alternately until a satisfactory result is obtained. The superior performance of the proposed algorithm in nonlinear spectral unmixing has been validated through experiments with both synthetic and real hyperspectral data, where traditional and state-of-the-art algorithms are compared.