Background: Colorectal cancer (CRC) is a complex disease with monogenic, polygenic and environmental risk factors. Polygenic risk scores (PRS) are being developed to identify high polygenic risk individuals. Due to differences in genetic background, PRS distributions vary by ancestry, necessitating calibration. Methods: We compared four calibration methods using the All of Us Research Program Whole Genome Sequence data for a CRC PRS previously developed in participants of European and East Asian ancestry. The methods contrasted results from linear models with A) the entire data set OR an ancestrally diverse training set AND B) covariates including principal components of ancestry OR admixture. Calibration with the training set adjusted the variance in addition to the mean. Results: All methods performed similarly within ancestry with OR(95% C.I.) per s.d. change in raw PRS: African 1.5(1.02,2.08), Admixed American 2.2(1.27,3.85), European 1.6(1.43,1.89), and Middle Eastern 1.1(0.71,1.63). Using admixture and an ancestrally diverse training set resulted in distributions closest to standard Normal with accurate upper tail frequencies, whereas the other combinations resulted in inconsistent tail frequencies. Conclusions: Training a calibration model on ancestrally diverse participants to adjust both the mean and variance, using admixture as covariates, was best at identifying patients at high polygenic risk. Although the PRS is valid for most ancestries, its performance varies by ancestry, even after calibration. More diverse datasets are required to further develop and validate the PRS.