The spiral membrane separator is a novel module proposed for reducing the concentration polarization and membrane fouling of membrane separation process. This membrane separation process benefits from dean vortices produced by centrifugal instability in enhancing fluid mass transfer. A numerical stimulation of this membrane separation is presented and used to analyze the fluid flow characteristics and thoroughly understand the separation mechanism. The numerical model consists of a spiral flow path with rectangular sector. As the simulation with infiltration, the fluid domain of the ceramic membrane tube was as a porous medium domain. The standard momentum equation, added with the momentum equation source term which composed of the viscosity loss term and the inertia loss term, are figured out through the experimental characterization. In the simulation of spiral membrane separation, the Dean secondary flow structure is identified and found to enhance fluid mass transfer and to increase the permeate flux. The critical unstable state of spiral membrane separation is accordingly De=246 without the flow permeation and De=863 with the permeation, where Dean vortices cause collisions and the mixing of fluid particles. Then in the case of permeation, the fluid in separator at De=1232 is numerically simulated to show that the higher flow velocity and a large fluctuating trend of wall shear stress near the membrane surface (inside), which mainly contributed to alleviate concentration polarization and membrane fouling.