Reinforcement Learning-powered Semantic Communication via Semantic Similarity

Abstract

We introduce a new semantic communication mechanism, whose key idea is to preserve the semantic information instead of strictly securing the bit-level precision. Starting by analyzing the defects of existing joint source channel coding (JSCC) methods, we show that the commonly used bit-level metrics are vulnerable of catching important semantic meaning and structures. To address this problem, we take advantage of learning from semantic similarity, instead of relying on conventional paired bit-level supervisions like cross entropy and bit error rate. However, to develop such a semantic communication system is indeed a nontrivial task, considering the nondifferentiability of most semantic metrics as well as the instability from noisy channels. To further resolve these issues, we put forward a reinforcement learning (RL)-based solution which allows us to simultaneously optimize any user-defined semantic measurement by using the policy gradient technique, and to interact with the surrounding noisy environment in a natural way. We have testified the proposed method in the challenging European-parliament dataset. Experiments on both AWGN and phase-invariant fading channel have confirmed the superiority of our method in revealing the semantic meanings, and better handling the channel noise especially in low-SNR situations. Apart from the experimental results, we further provide an indepth look at how the semantics model behaves, along with its superb generalization ability in real-life examples. As a brand new method in learning-based JSCC tasks, we also exemplify an RL-based image transmission paradigm, both to prove the generalization ability, and to leave this new topic for future discussion.

Common Language for Goal-Oriented Semantic Communications: A Curriculum Learning Framework

“…These matrices are generated according to the chain of events of each task. For comparison purposes, we use a traditional RL solution that performs random exploration between all subsets of B to solve the optimization in (5).…”

“…The work in [4] introduced a Bayesian game to minimize the end-to-end average semantic error. In [5], the authors proposed a reinforcement learning (RL) method to capture the meaning of transmitted information by learning se-mantic similarities between them. The work in [6] introduced a model for implementing semantic communications to address the reliability and latency requirements for drone networks.…”

Semantic Communication Meets Edge Intelligence

“…By jointly considering the accuracy and timeliness of information, the authors in [8] introduce the metric of age of incorrect information (AoII) into SemCom, which can measure the network performance by looking at the bigger picture of the packet's role in achieving the overall communication goal. Moreover, for the cases where the benefits of the packet content to be transmitted are evaluated to be important for the system objective, the value of the information (VoI) is of TABLE I: Some semantic metrics derived from NLP [7] Semantic metrics Advantages Drawbacks Bilingual evaluation understudy (BLEU) BLEU is a method for automatic evaluation for machine translation. It is used to is to compare word groups with different size of the candidate with that of the reference translation and count the number of matches.…”

“…2) Deep Reinforcement Learning based SE: Deep Reinforcement learning (DRL) can integrate the non-differentiable semantic metrics like BLEU into SE training. In the DRLbased SE for text transmission in [7], long short-term memory networks are employed in the encoder and decoder. The state is defined as the recurrent state of the decoder and the previously generated words.…”

Adaptive Bit Rate Control in Semantic Communication with Incremental Knowledge-based HARQ

“…[19] proposed using reinforcement learning (RL) to optimize semantic transmission. [20] developed a universal semantic communication system by optimizing the semantic similarity in an RL manner. [21] introduced a novel confidence-based JSCC scheme for semantic transmission.…”