My Boss is Really Cool: Malware-Induced Misperception in Workplace Communication Through Covert Linguistic Manipulation of Emails

"TL;DR:" Out-of-Context Adversarial Text Summarization and Hashtag Recommendation

“…In Section 7 we expand on the future adaptation for generating abstractive summaries, by using sentences that are different than the original sentences. This adaptation is possible given that approaches for automated word manipulation in various forms of online text such as email [47], Twitter posts [25], Wikipedia articles [46], or text converted to speech by an voice assistant [48] exist.…”

“…The intention to publish automated language models despite the adversarial output, after all, is to increase defenders' understanding of the threat from text generation, and contribute to building tools that could guard against the harms [27,41,65]. The Kerckhoffs' doctrine that "security through obscurity doesn't work" is thus thrown into the mix as an argumentation for publishing fake text generators because an adversary would figure out the language model themselves, if they haven't already, and proceed with exploitation (akin to zero-day vulnerabilities) [47]. Adversaries, in the traditional security sense, pose extensive knowledge of targeted systems that help them work through the obscurity or find a zeroday vulnerability.…”

“…Though not in an automated fashion, a similar attempt of out-ofcontext trolling was used by Robert F. Kennedy Jr. in his attempt to discredit the COVID-19 vaccination (which earned him a ban from Instagram) [6]. In this context, the misperception-inducing approach of word manipulation from [47], an adversary could also go further, for example, to append the first summary sentence with ..., which resulted in death in several cases in Florida in order to hint an extreme outcome of the vaccination. Certainly this moves the summary into the "fake news" category, and we provided this example to highlight the dangers of abstractive re-purposing of Out-of-Context Summarizer.…”

“…It's trivial to limit the summary and select one sentence to be a headline, for example, selecting the first sentence in 13 verbatim or modifying it to read CDC recommends that you not get a second shot of that vaccine, if you had a severe reaction after getting the first shot of a COVID-19 vaccine. An adversary could use the approach for modifying emails from [47] to simply make an email appear with a subject line CDC recommends that you not get a second shot of that vaccine. Similarly, one could use the approach from [48] and create a third-party COVID-19 skill that will deliver headlines to Amazon Alexa users by summarizing articles from the CDC website.…”