Reinforcement Learning for Nested Polar Code Construction

Abstract

In this paper, we model nested polar code construction as a Markov decision process (MDP), and tackle it with advanced reinforcement learning (RL) techniques. First, an MDP environment with state, action, and reward is defined in the context of polar coding. Specifically, a state represents the construction of an (N, K) polar code, an action specifies its reduction to an (N, K − 1) subcode, and reward is the decoding performance. A neural network architecture consisting of both policy and value networks is proposed to generate actions based on the observed states, aiming at maximizing the overall rewards. A loss function is defined to trade off between exploitation and exploration. To further improve learning efficiency and quality, an "integrated learning" paradigm is proposed. It first employs a genetic algorithm to generate a population of (sub-)optimal polar codes for each (N, K), and then uses them as prior knowledge to refine the policy in RL. Such a paradigm is shown to accelerate the training process, and converge at better performances. Simulation results show that the proposed learning-based polar constructions achieve comparable, or even better, performances than the state of the art under successive cancellation list (SCL) decoders. Last but not least, this is achieved without exploiting any expert knowledge from polar coding theory in the learning algorithms.

Decoding Reed-Muller Codes with Successive Codeword Permutations

“…11 that at a similar FER performance, the decoding latency of the proposed decoder is significantly smaller than that of the RPA decoder. In addition, the proposed decoder requires a computational complexity that is several orders of magnitude lower than that of the RPA decoder for RM (3,8) and RM (4,8). As seen in Table III, the proposed decoder requires lower memory consumption than the RPA decoder for all the considered RM codes of length 256 that reaches up to 85% for RM (4,8).…”

“…The selection criteria in (8), which is used in [16], is an oversimplification that does not take into account the existing parity constraints in the code. In fact, it treats all the constituent RM codes λ as Rate-1 codes.…”

“…Since ML decoding can be efficiently implemented for RM codes of order 0 or 1, the permutation metric M π α (λ) can be efficiently calculated if λ is a zero-order RM code (Rep code) or a first-order RM code. When λ is of order 2 or higher, we propose to simplify the computation of M π α (λ) by using (8). The calculation of M π α (λ) for the considered special nodes in this paper is summarized as follows.…”

“…11 illustrates the error-correction performance, decoding latency, and computational complexity of the proposed SFP-FSCL decoder compared with RPA-based decoding [18], [19] for the RM codes of length 256 with r ∈ {2, 3, 4}. The list size L of the SFP-FSCL decoder is set to 64 for RM(2, 8), 256 for RM (3,8), and 128 for RM(4, 8), to have a similar FER performance to the RPA decoder. In addition, the lower bound of the FER performance under ML decoding is also plotted as a reference.…”

“…RM codes are similar to polar codes under the factor-graph representation of the codes. The main difference between RM and polar codes is that RM codes are constructed to maximize the minimum distance among all the codewords [1], [2], while polar codes are constructed to minimize the error probability under successive-cancellation (SC) decoding [4]- [7] or SC list (SCL) decoding [8], [9]. An advantage of RM codes over polar codes is that the code construction of RM codes is channel independent, which does not impose additional complexity when the codes are constructed for different communication mediums, even under variable channel conditions.…”