Neural Linguistic Steganography

Abstract

Whereas traditional cryptography encrypts a secret message into an unintelligible form, steganography conceals that communication is taking place by encoding a secret message into a cover signal. Language is a particularly pragmatic cover signal due to its benign occurrence and independence from any one medium. Traditionally, linguistic steganography systems encode secret messages in existing text via synonym substitution or word order rearrangements. Advances in neural language models enable previously impractical generation-based techniques. We propose a steganography technique based on arithmetic coding with large-scale neural language models. We find that our approach can generate realistic looking cover sentences as evaluated by humans, while at the same time preserving security by matching the cover message distribution with the language model distribution.

Near-imperceptible Neural Linguistic Steganography via Self-Adjusting Arithmetic Coding

“…Traditionally, arithmetic coding encodes a string of elements into a bit sequence. To use such a coding for linguistic steganography, we follow (Ziegler et al, 2019) and reverse the encoding order. Namely, we encode a bit sequence (ciphertext) into a string of tokens (cover text) and decode a cover text to its original ciphertext.…”

“…Due to the above reasons, people in practice will truncate the LM distribution to include only top K most likely tokens (Ziegler et al, 2019), which leads to the following distribution:…”

“…Recent advances in neural language models (LMs) (Józefowicz et al, 2016;Radford et al, 2019;Yang et al, 2019a) have enabled a diagram shift from edit-based methods to generation-based methods which directly output a cover text by encoding the message reversibly in the choices of tokens. Various encoding algorithms (Fang et al, 2017;Yang et al, 2019b;Ziegler et al, 2019) have been proposed to leverage neural LMs to generate high-quality cover texts in terms of both fluency and information hiding capacity. However, most of the existing methods do not provide explicit guarantees on the imperceptibility of generated cover text (i.e., to what extent the cover text is indistinguishable from natural texts without hidden messages).…”

“…One recent exception is the work (Dai and Cai, 2019) which shows the imperceptibility of the method in Fang et al (2017). Nevertheless, for other more advanced steganography methods (Yang et al, 2019b;Ziegler et al, 2019), their imperceptibilities still remain unknown.…”

“…In this work, we propose a new linguistic steganography method with guaranteed imperceptibility. Our new method is built based on the previous study (Ziegler et al, 2019) which views each secret message as a binary fractional number and encodes it using arithmetic coding (Rissanen and Langdon, 1979) with a pretrained neural LM. This method generates cover text tokens one at a time (c.f.…”