Reinforcement learning (RL) based techniques have been employed for the tracking and adaptive cruise control of a small-scale vehicle with the aim to transfer the obtained knowledge to a full-scale intelligent vehicle in the near future. Unlike most other control techniques, the purpose of this study is to seek a practical method that enables the vehicle, in the real environment and in real time, to learn the control behavior on its own while adapting to the changing circumstances. In this context, it is necessary to design an algorithm that symmetrically considers both time efficiency and accuracy. Meanwhile, in order to realize adaptive cruise control specifically, a set of symmetrical control actions consisting of steering angle and vehicle speed needs to be optimized simultaneously. In this paper, firstly, the experimental setup of the small-scale intelligent vehicle is introduced. Subsequently, three model-free RL algorithm are conducted to develop and finally form the strategy to keep the vehicle within its lanes at constant and top velocity. Furthermore, a model-based RL strategy is compared that incorporates learning from real experience and planning from simulated experience. Finally, a Q-learning based adaptive cruise control strategy is intermixed to the existing tracking control architecture to allow the vehicle slow-down in the curve and accelerate on straightaways. The experimental results show that the Q-learning and Sarsa (λ) algorithms can achieve a better tracking behavior than the conventional Sarsa, and Q-learning outperform Sarsa (λ) in terms of computational complexity. The Dyna-Q method performs similarly with the Sarsa (λ) algorithms, but with a significant reduction of computational time. Compared with a fine-tuned proportion integration differentiation (PID) controller, the good-balanced Q-learning is seen to perform better and it can also be easily applied to control problems with over one control actions.
Obesity is closely associated with low-bone-mass disorder. Discoidin domain receptor 2 (DDR2) plays essential roles in skeletal metabolism, and is probably involved in fat metabolism. To test the potential role of DDR2 in fat and fat-bone crosstalk, Ddr2 conditional knockout mice (Ddr2Adipo) were generated in which Ddr2 gene is exclusively deleted in adipocytes by Adipoq Cre. We found that Ddr2Adipo mice are protected from fat gain on high-fat diet, with significantly decreased adipocyte size. Ddr2Adipo mice exhibit significantly increased bone mass and mechanical properties, with enhanced osteoblastogenesis and osteoclastogenesis. Marrow adipocyte is diminished in the bone marrow of Ddr2Adipo mice, due to activation of lipolysis. Fatty acid in the bone marrow was reduced in Ddr2Adipo mice. RNA-Seq analysis identified adenylate cyclase 5 (Adcy5) as downstream molecule of Ddr2. Mechanically, adipocytic Ddr2 modulates Adcy5-cAMP-PKA signaling, and Ddr2 deficiency stimulates lipolysis and supplies fatty acid for oxidation in osteoblasts, leading to the enhanced osteoblast differentiation and bone mass. Treatment of Adcy5 specific inhibitor abolishes the increased bone mass gain in Ddr2Adipo mice. These observations establish, for the first time, that Ddr2 plays an essential role in the crosstalk between fat and bone. Targeting adipocytic Ddr2 may be a potential strategy for treating obesity and pathological bone loss simultaneously.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.