The alteration of age‐related molecules in the bone marrow microenvironment is one of the driving forces in osteoporosis. These molecules inhibit bone formation and promote bone resorption by regulating osteoblastic and osteoclastic activity, contributing to age‐related bone loss. Here, we observed that the level of microRNA‐31a‐5p (miR‐31a‐5p) was significantly increased in bone marrow stromal cells (BMSCs) from aged rats, and these BMSCs demonstrated increased adipogenesis and aging phenotypes as well as decreased osteogenesis and stemness. We used the gain‐of‐function and knockdown approach to delineate the roles of miR‐31a‐5p in osteogenic differentiation by assessing the decrease of special AT‐rich sequence‐binding protein 2 (SATB2) levels and the aging of BMSCs by regulating the decline of E2F2 and recruiting senescence‐associated heterochromatin foci (SAHF). Notably, expression of miR‐31a‐5p, which promotes osteoclastogenesis and bone resorption, was markedly higher in BMSCs‐derived exosomes from aged rats compared to those from young rats, and suppression of exosomal miR‐31a‐5p inhibited the differentiation and function of osteoclasts, as shown by elevated RhoA activity. Moreover, using antagomiR‐31a‐5p, we observed that, in the bone marrow microenvironment, inhibition of miR‐31a‐5p prevented bone loss and decreased the osteoclastic activity of aged rats. Collectively, our results reveal that miR‐31a‐5p acts as a key modulator in the age‐related bone marrow microenvironment by influencing osteoblastic and osteoclastic differentiation and that it may be a potential therapeutic target for age‐related osteoporosis.