Neural Architecture Search for Fracture Classification

Pourchot, Aloïs; Bailly, Kévin; Ducarouge, Alexis; Sigaud, Olivier

doi:10.1109/icip46576.2022.9897533

Cited by 2 publications

(2 citation statements)

References 19 publications

(24 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This method inherits the credit assignment capa-bility of RL and effectively explores different sets of policies. In parallel, CEM-RL (Pourchot and Sigaud 2019) integrates Cross-Entropy Method (CEM) into Twin Delayed Deep Deterministic policy gradient algorithm (TD3) (Fujimoto, Hoof, and Meger 2018). CEM is an optimization algorithm used to solve stochastic optimization problems, which iteratively updates a population of candidate solutions by cross-entropy based on the given objective function.…”

Section: Related Workmentioning

confidence: 99%

ERL-TD: Evolutionary Reinforcement Learning Enhanced with Truncated Variance and Distillation Mutation

Lin,

Chen,

et al. 2024

AAAI

View full text Add to dashboard Cite

Recently, an emerging research direction called Evolutionary Reinforcement Learning (ERL) has been proposed, which combines evolutionary algorithm with reinforcement learning (RL) for tackling the tasks of sequential decision making. However, the recently proposed ERL algorithms often suffer from two challenges: the inaccuracy of policy estimation caused by the overestimation bias in RL and the insufficiency of exploration caused by inefficient mutations. To alleviate these problems, we propose an Evolutionary Reinforcement Learning algorithm enhanced with Truncated variance and Distillation mutation, called ERL-TD. We utilize multiple Q-networks to evaluate state-action pairs, so that multiple networks can provide more accurate evaluations for state-action pairs, in which the variance of evaluations can be adopted to control the overestimation bias in RL. Moreover, we propose a new distillation mutation to provide a promising mutation direction, which is different from traditional mutation generating a large number of random solutions. We evaluate ERL-TD on the continuous control benchmarks from the OpenAI Gym and DeepMind Control Suite. The experiments show that ERL-TD shows excellent performance and outperforms all baseline RL algorithms on the test suites.

show abstract

Section: Related Workmentioning

confidence: 99%

ERL-TD: Evolutionary Reinforcement Learning Enhanced with Truncated Variance and Distillation Mutation

Lin,

Chen,

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…While PBT-MAP-ELITES and PBT use similar strategies to update the population of agents, PBT only seeks the highest-performing agent by extracting the best one from the final population while PBT-MAP-ELITES aims to find a diverse collection of high-performing agents. Several methods such as CERL, ERL, and CEM-RL (Pourchot & Sigaud, 2019;Khadka & Tumer, 2018;Khadka et al, 2019) combine ES algorithms with PBRL methods to improve the asymptotic performance and sample efficiency of standard RL methods. Other methods, such as DvD (Parker-Holder et al, 2020) and P3S-TD3 (Jung et al, 2020), train populations of agents and add terms in their loss functions to encourage the agents to explore different regions of the state-action space but always with the end goal of maximizing the performance of the best agent in the population.…”

Section: Literature Reviewmentioning

confidence: 99%

Evolving Populations of Diverse RL Agents with MAP-Elites

Pierrot¹,

Flajolet²

2023

Preprint

View full text Add to dashboard Cite

Quality Diversity (QD) has emerged as a powerful alternative optimization paradigm that aims at generating large and diverse collections of solutions, notably with its flagship algorithm MAP-ELITES (ME) which evolves solutions through mutations and crossovers. While very effective for some unstructured problems, early ME implementations relied exclusively on random search to evolve the population of solutions, rendering them notoriously sample-inefficient for highdimensional problems, such as when evolving neural networks. Follow-up works considered exploiting gradient information to guide the search in order to address these shortcomings through techniques borrowed from either Black-Box Optimization (BBO) or Reinforcement Learning (RL). While mixing RL techniques with ME unlocked state-of-the-art performance for robotics control problems that require a good amount of exploration, it also plagued these ME variants with limitations common among RL algorithms that ME was free of, such as hyperparameter sensitivity, high stochasticity as well as training instability, including when the population size increases as some components are shared across the population in recent approaches. Furthermore, existing approaches mixing ME with RL tend to be tied to a specific RL algorithm, which effectively prevents their use on problems where the corresponding RL algorithm fails. To address these shortcomings, we introduce a flexible framework that allows the use of any RL algorithm and alleviates the aforementioned limitations by evolving populations of agents (whose definition include hyperparameters and all learnable parameters) instead of just policies. We demonstrate the benefits brought about by our framework through extensive numerical experiments on a number of robotics control problems, some of which with deceptive rewards, taken from the QD-RL literature. We open source an efficient JAX-based implementation of our algorithm in the QDax library 1 .

show abstract

Neural Architecture Search for Fracture Classification

Cited by 2 publications

References 19 publications

ERL-TD: Evolutionary Reinforcement Learning Enhanced with Truncated Variance and Distillation Mutation

ERL-TD: Evolutionary Reinforcement Learning Enhanced with Truncated Variance and Distillation Mutation

Evolving Populations of Diverse RL Agents with MAP-Elites

Contact Info

Product

Resources

About