The anisotropy of low energy (0∼5eV) optical excitations in strongly correlated transition-metal oxides is closely related to the spin and orbital orderings. The recent successes of LDA+U method in describing the magnetic and electronic structures enable us to calculate the optical conductivity from first-principles. The LaVO3 and YVO3, both of which have 3d 2 configuration and have various spin and orbital ordered phases at low temperature, show distinct anisotropy in the optical spectra. The effects of spin and orbital ordering on the anisotropy are studied in detail based on our first-principles calculations. The experimental spectra of both compounds at low temperature phases can be qualitatively explained with our calculations, while the studies for the intermediate temperature phase of YVO3 suggest the substantial persistence of the low temperature phase at elevated temperature.