Solving Multiple Isolated, Interleaved, and Blended Tasks through Modular Neuroevolution

Schrum, Jacob; Miikkulainen, Risto

doi:10.1162/evco_a_00181

Cited by 9 publications

(11 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We also demonstrated that evolution guided towards a single user-defined decomposition does not perform well for tasks that do not have a very obvious structure. This agrees with previous work demonstrating that evolving neural networks often end up with unexpected decomposition patterns not agreeing with human intuition (Huizinga et al, 2016;Schrum and Miikkulainen, 2016b;Ellefsen et al, 2015). The technique of guiding evolving neural networks towards a diversity of decomposition patterns presents a way to take advantage of unexpected, creative solutionsallowing an automatic way to discover many functional problem decompositions.…”

Section: Resultssupporting

confidence: 88%

“…Modularity in evolving neural networks has been demonstrated to improve performance on complex tasks, as it allows problem decomposition, hierarchical knowledge structures and multimodal behavior. There is therefore a growing interest in techniques for increasing the functional modularity of evolving neural networks (Clune et al, 2013;Mengistu and Clune, 2016;Schrum and Miikkulainen, 2016b;Velez and Clune, 2017). Most techniques for increasing modularity in neuroevolution belong to one of two extremes.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Guiding Neuroevolution with Structural Objectives

Ellefsen

Huizinga

Tørresen

2020

Evolutionary Computation

View full text Add to dashboard Cite

The structure and performance of neural networks are intimately connected, and by use of evolutionary algorithms, neural network structures optimally adapted to a given task can be explored. Guiding such neuroevolution with additional objectives related to network structure has been shown to improve performance in some cases, especially when modular neural networks are beneficial. However, apart from objectives aiming to make networks more modular, such structural objectives have not been widely explored. We propose two new structural objectives and test their ability to guide evolving neural networks on two problems which can benefit from decomposition into subtasks. The first structural objective guides evolution to align neural networks with a user-recommended decomposition pattern. Intuitively, this should be a powerful guiding target for problems where human users can easily identify a structure. The second structural objective guides evolution towards a population with a high diversity in decomposition patterns. This results in exploration of many different ways to decompose a problem, allowing evolution to find good decompositions faster. Tests on our target problems reveal that both methods perform well on a problem with a very clear and decomposable structure. However, on a problem where the optimal decomposition is less obvious, the structural diversity objective is found to outcompete other structural objectives -and this technique can even increase performance on problems without any decomposable structure at all.

show abstract

Section: Resultssupporting

confidence: 88%

Section: Introductionmentioning

confidence: 99%

Guiding Neuroevolution with Structural Objectives

Ellefsen

Huizinga

Tørresen

2020

Evolutionary Computation

View full text Add to dashboard Cite

show abstract

“…21,23-25 In contrast, artificial intelligence tools for structured data are available and have already surpassed human performance in many areas. 25-31 Of note, bioinformatics pipelines generate mostly structured, discrete data. Thus, we consider the inability of humans to gain access to the full potential of the pipeline output, coupled with the discrete nature of the data and the final binary reporting decision, as an ideal setting 32 to assess the performance of an artificial intelligence–based decision support system for variant reporting.…”

Section: Introductionmentioning

confidence: 99%

Artificial Intelligence Approach for Variant Reporting

Zomnir

Lipkin

Pacula

et al. 2018

JCO Clinical Cancer Informatics

View full text Add to dashboard Cite

Purpose Next-generation sequencing technologies are actively applied in clinical oncology. Bioinformatics pipeline analysis is an integral part of this process; however, humans cannot yet realize the full potential of the highly complex pipeline output. As a result, the decision to include a variant in the final report during routine clinical sign-out remains challenging. Methods We used an artificial intelligence approach to capture the collective clinical sign-out experience of six board-certified molecular pathologists to build and validate a decision support tool for variant reporting. We extracted all reviewed and reported variants from our clinical database and tested several machine learning models. We used 10-fold cross-validation for our variant call prediction model, which derives a contiguous prediction score from 0 to 1 (no to yes) for clinical reporting. Results For each of the 19,594 initial training variants, our pipeline generates approximately 500 features, which results in a matrix of > 9 million data points. From a comparison of naive Bayes, decision trees, random forests, and logistic regression models, we selected models that allow human interpretability of the prediction score. The logistic regression model demonstrated 1% false negativity and 2% false positivity. The final models’ Youden indices were 0.87 and 0.77 for screening and confirmatory cutoffs, respectively. Retraining on a new assay and performance assessment in 16,123 independent variants validated our approach (Youden index, 0.93). We also derived individual pathologist-centric models (virtual consensus conference function), and a visual drill-down functionality allows assessment of how underlying features contributed to a particular score or decision branch for clinical implementation. Conclusion Our decision support tool for variant reporting is a practically relevant artificial intelligence approach to harness the next-generation sequencing bioinformatics pipeline output when the complexity of data interpretation exceeds human capabilities.

show abstract

“…However, after activating certain power-ups, the ghosts become vulnerable for a brief period of time. The agent can consume these ghosts for a score boost.The switch in ghost dynamics necessitates a change in the game-play strategy, since multiple distinct modes of behavior are required under different conditions [4,5]. Despite the need for multi-modal behaviors, conventional reinforcement-learning approaches have focused on constructing monolithic policies.…”

mentioning

confidence: 99%

Analysis of Agent Expertise in Ms. Pac-Man Using Value-of-Information-Based Policies

Sledge

Prı́ncipe

2019

IEEE Trans. Games

View full text Add to dashboard Cite

Conventional reinforcement learning methods for Markov decision processes rely on weakly-guided, stochastic searches to drive the learning process. It can therefore be difficult to predict what agent behaviors might emerge. In this paper, we consider an information-theoretic cost function for performing constrained stochastic searches that promote the formation of risk-averse to risk-favoring behaviors. This cost function is the value of information, which provides the optimal trade-off between the expected return of a policy and the policy's complexity; policy complexity is measured by number of bits and controlled by a single hyperparameter on the cost function. As the policy complexity is reduced, the agents will increasingly eschew risky actions. This reduces the potential for high accrued rewards. As the policy complexity increases, the agents will take actions, regardless of the risk, that can raise the long-term rewards. The obtainable reward depends on a single, tunable hyperparameter that regulates the degree of policy complexity.We evaluate the performance of value-of-information-based policies on a stochastic version of Ms. Pac-Man. A major component of this paper is the demonstration that ranges of policy complexity values yield different game-play styles and explaining why this occurs. We also show that our reinforcementlearning search mechanism is more efficient than the others we utilize. This result implies that the value of information theory is appropriate for framing the exploitation-exploration trade-off in reinforcement learning.Index Terms-Value of information, constrained search, reinforcement learning, information theory Isaac J. Sledge is with the 2 objective of the agent is to clear the environment of pellets while navigating around the ghosts. However, after activating certain power-ups, the ghosts become vulnerable for a brief period of time. The agent can consume these ghosts for a score boost.The switch in ghost dynamics necessitates a change in the game-play strategy, since multiple distinct modes of behavior are required under different conditions [4,5]. Despite the need for multi-modal behaviors, conventional reinforcement-learning approaches have focused on constructing monolithic policies. Such policies would implement the same agent behaviors regardless of the vulnerability of the ghosts. Although it is possible to represent multimodal behavior with these policies, it can be difficult to learn such behavior. This is, in part, due to risk. For instance, throughout the learning process, an agent may have learned to avoid colliding with the ghosts. Without straying from this behavior, the agent will not learn that there are instances where it can safely chase the ghosts.In this paper, we consider an information-theoretic learning [6] approach for performing constrained stochastic searches that promote a continuum of risk-averse to risk-favoring agent behaviors during reinforcement learning. This, in turn, leads to a principled exploration of the state-action space that aids in the...

show abstract

Solving Multiple Isolated, Interleaved, and Blended Tasks through Modular Neuroevolution

Cited by 9 publications

References 41 publications

Guiding Neuroevolution with Structural Objectives

Guiding Neuroevolution with Structural Objectives

Artificial Intelligence Approach for Variant Reporting

Analysis of Agent Expertise in Ms. Pac-Man Using Value-of-Information-Based Policies

Contact Info

Product

Resources

About