Towards Near-imperceptible Steganographic Text

Abstract

We show that the imperceptibility of several existing linguistic steganographic systems (Fang et al., 2017;Yang et al., 2018) relies on implicit assumptions on statistical behaviors of fluent text. We formally analyze them and empirically evaluate these assumptions. Furthermore, based on these observations, we propose an encoding algorithm called patient-Huffman with improved near-imperceptible guarantees.

Frustratingly Easy Edit-based Linguistic Steganography with a Masked Language Model

“…We found that 1.41% of the masked tokens had substitution candidates that did not reproduce the original segmentations. Although this danger applies equally to generation-based steganography built on top of subword LMs (Dai and Cai, 2019;Ziegler et al, 2019;Shen et al, 2020), to our knowledge, we are the first to point it out.…”

“…Let n be the largest integer that satisfies 2 n ≤ c, where c is the number of the remaining items. Each item is given a unique bit chunk of size n. Coding is an active research topic (Dai and Cai, 2019;Ziegler et al, 2019;Shen et al, 2020) and is orthogonal to our core proposal.…”

“…With advances in neural language models (LMs), edit-based approaches have been replaced by generation-based ones (Fang et al, 2017;Yang et al, 2019;Dai and Cai, 2019;Ziegler et al, 2019;Shen et al, 2020). In these approaches, bit chunks are directly assigned to the conditional probability distribution over the next word estimated by the LM, yielding impressive payload capacities of 1-5 bits per word (Shen et al, 2020).…”

Provably Secure Generative Linguistic Steganography

“…Then they took the token which has the same code as the secret information. Dai and Cai (2019) proposed patient-Huffman, which was an improved version of Yang et al (2018a) that sacrificed embedding capacity for imperceptibility. They first calculated the distortion (total variation distance or KL divergence) between q and p LM and then only used Huffman coding embedding algorithm to embed secret information when the distortion was less than a preset threshold δ.…”

“…In recent years, powered by the advanced technology of deep learning and natural language processing, language models based on neural networks have made significant progress in generating fluent text (Radford et al, 2019;Brown et al, 2020), which bring new vitality to linguistic steganography and facilitate the investigation of generationbased methods (Fang et al, 2017;Yang et al, 2018a;Dai and Cai, 2019;Ziegler et al, 2019;Yang et al, 2020a;Zhou et al, 2021). The generative linguistic steganography directly transform secret information into innocuous-looking steganographic text (stegotext) without any covertext.…”