To explore the effect of non-uniform polarization on orbital angular momentum (OAM) in anisotropic media, we investigate the evolution of the spiral spectra and OAM densities of non-uniformly polarized vortex (NUPV) beams in uniaxial crystals propagating orthogonal to the optical axis, where the case of uniformly polarized vortex (UPV) beams with left-handed elliptical polarization is also investigated. At the input plane, the NUPV beams present their spiral spectra of m concentrated at m=l±1 rather than m=l, and reveal the relation between topological charge l, mode of spiral spectra m and the power weight value R
m
expressed by $l = \sum\limits_{m = - \infty }^\infty {m{R_m}} $. The relation $l = \sum\limits_{m = - \infty }^\infty {m{R_m}} $ is still satisfied for UPV beams in uniaxially anisotropic crystals, whereas for NUPV beams their relations are no longer valid owing to non-uniform polarization. Furthermore, the analysis indicates that the asymmetrical distribution of power weight of spiral spectra and the non-zero value in the sum of longitudinal OAM densities originate from the initial non-uniform polarization and anisotropy in uniaxial crystals rather than topological charges. In addition, the relation between spiral spectrum and longitudinal OAM density is numerically discussed. This work may provide an avenue for OAM-based communications, optical metrology and imaging by varying the initial non-uniform polarization.