Ginkgo biloba is worldwide an economically valuable tree. The species has nearly got extinct during the Quaternary which likely result in reduction of its genetic variability. This is now conserved in few natural populations in China and number of cultivars which are, however, derived from a few ancient trees, helping the species survival in China through medieval times. Despite the recent interest on ginkgo, however, the detail knowledge of its genetic diversity, conserved in cultivated trees and cultivars remained poor. This limit efficient conservation of its diversity as well as efficient use of the existing germplasm resources. Here we performed genotyping-by-sequencing (GBS) sequencing on 102 ginkgo cultivated germplasm collected to explore their genetic structure, kinship and inbreeding prediction. And for the first time in ginkgo, genome-wide association analysis study (GWAS) was used to try gene mapping of seed traits. The results showed that the most of the germplasm did not show obvious genetic relationship. The size of ginkgo germplasm population expanded significantly around 1500 years ago during the Sui and Tang dynasties. Classification of seed cultivars based on phylogenetic perspective does not support the current classification criteria based on phenotype. Twenty-four candidate genes were localized after performing GWAS on the seed traits. Overall, this study reveal the genetic basis of ginkgo seed traits and shed insights into its cultivation history, these findings will facilitate the conservation and utilization of the domesticated germplasms of this living fossil plants.