Alzheimer's disease (AD) is a major risk for the aging population. The pathological hallmarks of AD, an abnormal deposition of amyloid β-protein (Aβ) and phosphorylated tau (pTau), have been demonstrated in the retinas of AD patients, including in prodromal patients with mild cognitive impairment (MCI). Aβ pathology, especially the accumulation of the amyloidogenic 42-residue long alloform (Aβ42), is considered an early and specific sign of AD, and together with tauopathy, confirms AD diagnosis. To visualize retinal Aβ and pTau, state-of-the-art methods use fluorescence. However, administering contrast agents complicates the imaging procedure. To address this problem, we developed a label-free hyperspectral imaging method to detect the spectral signatures of Aβ42 and pS396-Tau and predicted their abundance in retinal cross sections. For the first time, we reported the spectral signature of pTau and demonstrated an accurate prediction of Aβ and pTau distribution powered by deep learning. We expect our finding will lay the groundwork for label-free detection of AD at its very earliest roots.