Wavelength scanning interferometry (WSI) is a promising imaging technique for measuring the three-dimensional structure of the inner tissues in biomedicine. One of the important problems in WSI is how to estimate the number of layers without prior knowledge of the tissues. In this paper, a subspace decomposition-based virtual multichannel (SDVM) algorithm has been proposed for this problem, in which the interference signal of each pixel is first cast into a virtual multichannel model by the sum-to-product formula of trigonometric functions, and then, by combining the subspace decomposition method of multichannel-signal processing, the layers of the sample can be estimated in an unsupervised manner. The experimental results illustrate that SDVM is a robust algorithm with a success rate greater than 85%.