This study investigated the nonlinear frequency conversions between the six polarization modes of a two-mode birefringent fiber. The aim was to demonstrate that the selective excitation of different combinations of linearly polarized spatial modes at the pump wavelength initiates distinct intermodal vector four-wave mixing processes. In particular, this study shows that exciting two orthogonally polarized LP 01 and LP 11 modes can result in simultaneous generation of up to three pairs of different spatial modes of orthogonal polarizations at different wavelengths. The role of the phase birefringence of the spatial modes in the phase-matching of such a four-wave mixing process was explained. Moreover, the theoretical predictions were verified through numerical simulations based on coupled nonlinear Schrodinger equations and also confirmed experimentally in a commercially available birefringent fiber.