Temperature Control for Automated Tape Laying with Infrared Heaters Based on Reinforcement Learning

Römer, Martin; Bergers, Johannes; Gabriel, Felix; Dröder, Klaus

doi:10.3390/machines10030164

Cited by 9 publications

(4 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As an internal measure of learning, similar to Römer et al (2022), we recorded the softmax probability for each Mean and std error (SEM) (shaded area) for the probability of actions that EP-BP and backpropagation (BP) models takes on CartPole-v0.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Combining backpropagation with Equilibrium Propagation to improve an Actor-Critic reinforcement learning framework

Kubo

Chalmers

Luczak

2022

Front. Comput. Neurosci.

View full text Add to dashboard Cite

Backpropagation (BP) has been used to train neural networks for many years, allowing them to solve a wide variety of tasks like image classification, speech recognition, and reinforcement learning tasks. But the biological plausibility of BP as a mechanism of neural learning has been questioned. Equilibrium Propagation (EP) has been proposed as a more biologically plausible alternative and achieves comparable accuracy on the CIFAR-10 image classification task. This study proposes the first EP-based reinforcement learning architecture: an Actor-Critic architecture with the actor network trained by EP. We show that this model can solve the basic control tasks often used as benchmarks for BP-based models. Interestingly, our trained model demonstrates more consistent high-reward behavior than a comparable model trained exclusively by BP.

show abstract

Section: Resultsmentioning

confidence: 99%

“…As an internal measure of learning, similar to Römer et al (2022) , we recorded the softmax probability for each action executed by the Actor network throughout learning. For each episode, we saved the probabilities of actions that EP-BP and BP took.…”

Section: Resultsmentioning

confidence: 99%

Combining backpropagation with Equilibrium Propagation to improve an Actor-Critic reinforcement learning framework

Kubo

Chalmers

Luczak

2022

Front. Comput. Neurosci.

View full text Add to dashboard Cite

show abstract

“…This tells us that our model is more stable than the base line model. As an internal measure of learning, similar to Römer et al [30], we recorded the softmax probability for each action executed by the Actor network throughout learning. For each episode, we saved the probabilities of actions that EP-BP and BP took.…”

Section: Methodsmentioning

confidence: 99%

Combining Backpropagation with Equilibrium Propagation to improve an Actor-Critic RL framework

Kubo

Chalmers

Luczak

2022

Preprint

View full text Add to dashboard Cite

Backpropagation has been used to train neural networks for many years, allowing them to solve a wide variety of tasks like image classification, speech recognition, and reinforcement learning tasks. But the biological plausibility of backpropagation as a mechanism of neural learning has been questioned. Equilibrium Propagation (EP) has been proposed as a more biologically plausible alternative and achieves comparable accuracy on the CIFAR-10 image classification task. This study proposes the first EP-based reinforcement learning architecture: an actor-critic architecture with the actor network trained by EP. We show that this model can solve the basic control tasks often used as benchmarks for BP-based models. Interestingly, our trained model demonstrates more consistent high-reward behavior than a comparable model trained exclusively by backpropagation.

show abstract

“…Compared with a hotplate or Peltier elements, microheaters with thermal mass and power consumption reduced, are more effective for portable applications [ 1 ]. Microheaters have been widely applied in various applications including gas sensors [ 2 , 3 ], microcalorimeters [ 4 ], gas flow meters [ 5 ], infrared sources [ 6 , 7 ], and thermal management [ 8 ]. Various structure designs of microheaters have been proposed for different applications.…”

Section: Introductionmentioning

confidence: 99%

Design and Thermal Analysis of Flexible Microheaters

Ruan

Chen

et al. 2022

Micromachines

View full text Add to dashboard Cite

With the development of flexible electronics, flexible microheaters have been applied in many areas. Low power consumption and fast response microheaters have attracted much attention. In this work, systematic thermal and mechanical analyses were conducted for a kind of flexible microheater with two different wire structures. The microheater consisted of polyethylene terephthalate (PET) substrate and copper electric wire with graphene thin film as the middle layer. The steady-state average temperature and heating efficiency for the two structures were compared and it was shown that the S-shaped wire structure was better for voltage-controlled microheater other than circular-shaped structure. In addition, the maximum thermal stress for both structures was from the boundary of microheaters, which indicated that not only the wire structure but also the shape of micro heaters should be considered to reduce the damage caused by thermal stress. The influence resulting from the thickness of graphene thin film also has been discussed. In all, the heating efficiency for flexible microheaters can be up to 135 °C/W. With the proposed PID voltage control system, the response time for the designed microheater was less than 10 s. Moreover, a feasible fabrication process flow for these proposed structures combing thermal analysis results in this work can provide some clues for flexible microheaters design and fabrication in other application areas.

show abstract

Temperature Control for Automated Tape Laying with Infrared Heaters Based on Reinforcement Learning

Cited by 9 publications

References 13 publications

Combining backpropagation with Equilibrium Propagation to improve an Actor-Critic reinforcement learning framework

Combining backpropagation with Equilibrium Propagation to improve an Actor-Critic reinforcement learning framework

Combining Backpropagation with Equilibrium Propagation to improve an Actor-Critic RL framework

Design and Thermal Analysis of Flexible Microheaters

Contact Info

Product

Resources

About