As a new member of 2D materials, GeSe has attracted considerable attention recently due to its fascinating in-plane anisotropic vibrational, electrical, and optical properties originating from the low-symmetry crystal structure. Among these anisotropic properties, the anisotropic optical property, as a new degree of freedom to manipulate optoelectronic properties in 2D materials, is of great importance for practical applications. However, the fundamental understanding of the optical anisotropy of GeSe is still under exploration, severely restricting its utility in polarization-sensitive optical systems. Here, a systematic study about the in-plane optical anisotropy of GeSe is reported, including its anisotropic optical absorption, reflection, extinction, and refraction. The anisotropic band structure of GeSe is experimentally observed for the first time through angle-resolved photoemission spectroscopy, explaining the origin of the optical anisotropy. The anisotropic reflection and refraction of GeSe are further directly visualized through the angle-dependent optical contrast of GeSe flakes by azimuth-dependent reflectance difference microscopy and polarization-resolved optical microscopy, respectively. Finally, GeSe-based photodetectors exhibit a polarization-sensitive photoresponsivity due to the intrinsic linear dichroism. This study provides fundamental information for the optical anisotropy of GeSe, forcefully stimulating the exploration of novel GeSe-based optical and optoelectronic applications.