Caucasian clover is the only perennial herb of the genus Leguminous clover with underground rhizomes. However, we know very little about its development process and mechanism. Transcriptome studies were conducted on the roots of Caucasian clover without a rhizome (NR) at the young seedling stage and the fully developed rhizome, including the root neck (R1), main root (R2), horizontal root (R3), and rhizome bud (R4), of the tissues in the mature phase. Compared with the rhizome in the mature phase, NR had 893 upregulated differentially expressed genes (DEGs), most of which were enriched in ‘phenylpropanoid biosynthesis’, ‘phenylalanine metabolism’, ‘DNA replication’ and ‘biosynthesis of amino acids’. A higher number of transcription factors (AP2/ERF, C2H2 and FAR1) were found in NR. There were highly expressed genes for R4, such as auxin response factor SAUR, galacturonosyltransferase (GAUT), and sucrose synthase (SUS). Phenylpropanoids are very important for the entire process of rhizome development. We drew a cluster heat map of genes related to the phenylpropanoid biosynthesis pathway, in which the largest number of genes belonged to COMT, and most of them were upregulated in R4.