Deep Reinforcement Learning for Soft, Flexible Robots: Brief Review with Impending Challenges

Bhagat, Sarthak; Banerjee, Hritwick; Tse, Zion Tsz Ho; Ren, Hongliang

doi:10.3390/robotics8010004

Cited by 94 publications

(57 citation statements)

References 112 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Even though the paper is successful for delay minimization, the other important factors, e.g., energy consumption optimization and offloading, are not discussed in this paper. Furthermore, Bhagat et al designed a self-sufficient intelligent agent that can be learned based on the information collected from the agent's environment by combining DRL algorithms and soft bio-inspired structures [16]. They also presented several examples in various real-world scenarios.…”

Section: Related Work: Reinforcement Learning For Mobile Edge Computingmentioning

confidence: 99%

Adaptive Real-Time Offloading Decision-Making for Mobile Edges: Deep Reinforcement Learning Framework and Simulation Results

et al. 2020

View full text Add to dashboard Cite

This paper proposes a novel dynamic offloading decision method which is inspired by deep reinforcement learning (DRL). In order to realize real-time communications in mobile edge computing systems, an efficient task offloading algorithm is required. When the decision of actions (offloading enabled, i.e., computing in clouds or offloading disabled, i.e., computing in local edges) is made by the proposed DRL-based dynamic algorithm in each unit time, it is required to consider real-time/seamless data transmission and energy-efficiency in mobile edge devices. Therefore, our proposed dynamic offloading decision algorithm is designed for the joint optimization of delay and energy-efficient communications based on DRL framework. According to the performance evaluation via data-intensive simulations, this paper verifies that the proposed dynamic algorithm achieves desired performance. IntroductionAccording to the fact that the 5G era has been realized based on networking technology innovation, many new computing and communications paradigms are introduced such as millimeter-wave communications, hyper-dense networks, and device-to-device proximal networking [1-3]. Among them, mobile edge computing (MEC) is one of the major technologies for realizing data/computing distribution in order to improve cellular network performance [4]. Based on the benefits of MEC technologies such as data rate improvement and quality enhancement, mobile cellular users can enjoy high quality, seamless, and real-time communication networking services.Together with the MEC technologies, many networked components are also of interest such as cloud servers and connected devices/vehicles. As the number of connected devices/vehicles increases, the amount of data transmitted to the MEC is also rapidly increasing. This obviously introduces serious network limitations such as data processing performance limits, storage capacity limits, and the battery use of terminal devices. Under the consideration of these limitations and problems, the use of cloud computing server is more efficient to deal with big data (gathered from connected vehicles/devices via MEC edges) than the local computing on terminals. On the other hand, for any cases where the data cannot be handled in the cloud servers due to delay requirements and security reasons, MEC devices should be able to handle or process the data from the connected vehicles/devices. Therefore, we can observe the trade-off between cloud computing servers and MEC servers (local edge computing servers). In summary, cloud servers have more power and centralized computing benefits comparing to MEC servers, whereas, it may have limitations when we have delay requirements and security requirements/regulations. Figure 1 shows the combined architecture of local edge computing and cloud computing. As the transmission from connected vehicles/devices (i.e., data upload and download) delay goes down, it is much more suitable for real-time processing. It enables real-time data processing and transmission using high-quality ...

show abstract

Section: Related Work: Reinforcement Learning For Mobile Edge Computingmentioning

confidence: 99%

Adaptive Real-Time Offloading Decision-Making for Mobile Edges: Deep Reinforcement Learning Framework and Simulation Results

et al. 2020

View full text Add to dashboard Cite

show abstract

“…In Table 2 of this paper [1], the caption was revised with the permission from the publishers as "SoRo applied to achieve state-of-the-art results alongside sub-domains where its utilization with deep reinforcement learning (DRL) and imitation learning techniques presently occur. Pictures adapted with permission from [40,41].…”

mentioning

confidence: 99%

“…In Figure In Figure 5 of this paper [1], the caption was revised with the permission from the publishers as "Expected application of DRL techniques in the task of navigation. Inset adapted with permission from [72].…”

mentioning

confidence: 99%

“…In Figure 7 of this paper [1], the caption was revised with the permission from the publishers as "Soft Robot Simulation on SOFA using Soft-robotics toolkit.…”

mentioning

confidence: 99%

“…In Table 4 of this paper [1], the caption was revised with the permission from the publishers as, "Instances of bio-inspired soft robotics applications that make use DRL The changes do not affect the scientific results. The manuscript will be updated and the original will remain online on the article webpage, with a reference to this Correction.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Correction: Bhagat, S.; et al. Deep Reinforcement Learning for Soft, Flexible Robots: Brief Review with Impending Challenges. Robotics 2019, 8, 4

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

Accelerating Auxetic Metamaterial Design with Deep Learning

Wilt

Yang

2020

Adv Eng Mater

View full text Add to dashboard Cite

Metamaterials can be designed to contain functional gradients with negative Poisson's ratio (NPR) that have counterintuitive behavior compared with monolithic materials. These NPR materials, referred to as auxetics, are relevant to engineering sciences because of their unique mechanical expansion. Previous studies have explored compliant actuators using analytical and numerically derived mechanics of materials principles. However, the control of compliant gradient mechanisms frequently uses complex analytical equations combined with traditional control algorithms, making them difficult to design. To confront the design processes and computational load, herein, machine learning is used to predict errors in compliant auxetic designs based on a mathematically optimal deformation. Finite element analysis and experimental specimens validate the theoretical mechanical behavior of a specific auxetic configuration as well as demonstrate the capabilities of additive manufacturing of graded auxetic materials. Pseudorandomized images and their respective computational deformation results are used to train a regressive model and predict the deviation from optimal behavior. The model predicts the deviation from the desired behavior with a mean average percent error below 5% for the validation set. Subsequently, a scalable workflow design process connecting the unique performance of auxetics to machine learning design predictions is proposed.

show abstract

Deep Reinforcement Learning for Soft, Flexible Robots: Brief Review with Impending Challenges

Cited by 94 publications

References 112 publications

Adaptive Real-Time Offloading Decision-Making for Mobile Edges: Deep Reinforcement Learning Framework and Simulation Results

Adaptive Real-Time Offloading Decision-Making for Mobile Edges: Deep Reinforcement Learning Framework and Simulation Results

Correction: Bhagat, S.; et al. Deep Reinforcement Learning for Soft, Flexible Robots: Brief Review with Impending Challenges. Robotics 2019, 8, 4

Accelerating Auxetic Metamaterial Design with Deep Learning

Contact Info

Product

Resources

About