Redesigning the Eterna100 for the Vienna 2 folding engine

Abstract

The rational design of RNA is becoming important for rapidly developing technologies in medicine and biochemistry. Recent work has led to the development of several RNA secondary structure design algorithms and corresponding benchmarks to evaluate their performance. However, the performance of these algorithms is linked to the nature of the underlying algorithms for predicting secondary structure from sequences. Here, we show that an online community of RNA design experts is capable of modifying an existing RNA secondary structure design benchmark (Eterna100) with minimal alterations to address changes in the folding engine used (Vienna 1.8 updated to Vienna 2.4). We tested this new Eterna100-V2 benchmark with five RNA design algorithms, and found that neural network-based methods exhibited reduced performance in the folding engine they were evaluated on in their respective papers. We investigated this discrepancy, and determined that structural features, previously classified as difficult, may be dependent on parameters inherent to the RNA energy function itself. These findings suggest that for optimal performance, future algorithms should focus on finding strategies capable of solving RNA secondary structure design benchmarks independently of the free energy benchmark used. Eterna100-V1 and Eterna100-V2 benchmarks and example solutions are freely available at https://github.com/eternagame/eterna100-benchmarking.

An evolutionary algorithm for inverse RNA folding inspired by Lévy flights

“…A new benchmark was performed on with achieving a 93% success rate. Compared to recently reported benchmark results (34), achieved similar performance to on : one target was unsolved by all existing tools and one target solved only by remained unsolved by .…”

“…The dataset (34) is available in two versions and both contain a set of 100 target structures extracted from the puzzle game and classified by their degree of difficulty. The was initially designed using 1.8.5, which relies on Turner1999 energy parameters (35).…”

“…Out of the 100 target secondary structures, 19 turned out to be unsolvable using the recent version of (Version 2.14). Subsequently, an (34) was released in which the 19 targets were slightly modified to be solvable using .…”

Differentiable Partition Function Calculation for RNA

“…Additionally, two of these four cases (i.e. puzzles 57 and 88) are known to be unsolvable using the Vienna 2 parameters [26].…”

Partial RNA Design

“…We, therefore, evaluate libLEARNA's first shot performance against Meta-LEARNA, the best performing approach of the LEARNA algorithms proposed by Runge et al [2019], on the 100 tasks of the Eterna100 benchmark version 2 [Koodli et al, 2021]. Figure 3 compares the average Hamming distances of the first predictions on each of the tasks between Meta-LEARNA and libLEARNA.…”