How Much Information Does a Human Translator Add to the Original?

Zoph, Barret; Ghazvininejad, Marjan; Knight, Kevin

doi:10.18653/v1/d15-1105

Cited by 4 publications

(16 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Hutter Prize [12], a competition to compress a 100 m-word extract of English Wikipedia, was designed to futher encourage research in text compression. Bilingual and multilingual text compression is a less-studied field [1,[13][14][15][16][17][18]. These papers provide different algorithms for compressing text in multilingual format, but they do not demonstrate how humans perform on this task.…”

Section: Related Workmentioning

confidence: 99%

“…Shannon reported a lower bound of 0.6 bpc. Table 1 gives the context for our work, drawing prior numbers from Zoph et al [1]. By introducing results from a Bilingual Shannon Game, we show that there is significant room for improving bilingual compression algorithms, meaning there is significant unexploited redundancy in translated texts.…”

Section: Introductionmentioning

confidence: 99%

“…Zoph et al [1] ask the question "How much information does a human translator add to the original? " That is, once a source text has been compressed, how many additional bits are required to encode its human translation?…”

Section: Introductionmentioning

confidence: 99%

“…To get an upper bound on this figure, Zoph et al [1] devise and implement an algorithm to actually compress target English in the presence of source Spanish. The size of their compressed English is 68% of the size of compressed Spanish.…”

Section: Introductionmentioning

confidence: 99%

“…This bound seems rather generous. In setting up a common task, Zoph et al [1] encourage researchers to develop improved bilingual compression technology [2].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Humans Outperform Machines at the Bilingual Shannon Game

Ghazvininejad¹,

Knight²

2016

Entropy

Self Cite

View full text Add to dashboard Cite

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

“…This bound seems rather generous. In setting up a common task, Zoph et al [1] encourage researchers to develop improved bilingual compression technology [2].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Humans Outperform Machines at the Bilingual Shannon Game

Ghazvininejad¹,

Knight²

2016

Entropy

Self Cite

View full text Add to dashboard Cite

show abstract

Confidence-Aware Scheduled Sampling for Neural Machine Translation

Liu¹,

Meng

Chen

et al. 2021

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

View full text Add to dashboard Cite

Scheduled sampling is an effective method to alleviate the exposure bias problem of neural machine translation. It simulates the inference scene by randomly replacing groundtruth target input tokens with predicted ones during training. Despite its success, its critical schedule strategies are merely based on training steps, ignoring the real-time model competence, which limits its potential performance and convergence speed. To address this issue, we propose confidence-aware scheduled sampling. Specifically, we quantify realtime model competence by the confidence of model predictions, based on which we design fine-grained schedule strategies. In this way, the model is exactly exposed to predicted tokens for high-confidence positions and still ground-truth tokens for low-confidence positions. Moreover, we observe vanilla scheduled sampling suffers from degenerating into the original teacher forcing mode since most predicted tokens are the same as ground-truth tokens. Therefore, under the above confidenceaware strategy, we further expose more noisy tokens (e.g., wordy and incorrect word order) instead of predicted ones for high-confidence token positions. We evaluate our approach on the Transformer and conduct experiments on large-scale WMT 2014 English-German, WMT 2014 English-French, and WMT 2019 Chinese-English. Results show that our approach significantly outperforms the Transformer and vanilla scheduled sampling on both translation quality and convergence speed.

show abstract

Neural Linguistic Steganography

Ziegler¹,

Deng²,

Rushton³

2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

Whereas traditional cryptography encrypts a secret message into an unintelligible form, steganography conceals that communication is taking place by encoding a secret message into a cover signal. Language is a particularly pragmatic cover signal due to its benign occurrence and independence from any one medium. Traditionally, linguistic steganography systems encode secret messages in existing text via synonym substitution or word order rearrangements. Advances in neural language models enable previously impractical generation-based techniques. We propose a steganography technique based on arithmetic coding with large-scale neural language models. We find that our approach can generate realistic looking cover sentences as evaluated by humans, while at the same time preserving security by matching the cover message distribution with the language model distribution.

show abstract

How Much Information Does a Human Translator Add to the Original?

Cited by 4 publications

References 29 publications

Humans Outperform Machines at the Bilingual Shannon Game

Humans Outperform Machines at the Bilingual Shannon Game

Confidence-Aware Scheduled Sampling for Neural Machine Translation

Neural Linguistic Steganography

Contact Info

Product

Resources

About