Deepfakes Generation and Detection: A Short Survey

Abstract

Advancements in deep learning techniques and the availability of free, large databases have made it possible, even for non-technical people, to either manipulate or generate realistic facial samples for both benign and malicious purposes. DeepFakes refer to face multimedia content, which has been digitally altered or synthetically created using deep neural networks. The paper first outlines the readily available face editing apps and the vulnerability (or performance degradation) of face recognition systems under various face manipulations. Next, this survey presents an overview of the techniques and works that have been carried out in recent years for deepfake and face manipulations. Especially, four kinds of deepfake or face manipulations are reviewed, i.e., identity swap, face reenactment, attribute manipulation, and entire face synthesis. For each category, deepfake or face manipulation generation methods as well as those manipulation detection methods are detailed. Despite significant progress based on traditional and advanced computer vision, artificial intelligence, and physics, there is still a huge arms race surging up between attackers/offenders/adversaries (i.e., DeepFake generation methods) and defenders (i.e., DeepFake detection methods). Thus, open challenges and potential research directions are also discussed. This paper is expected to aid the readers in comprehending deepfake generation and detection mechanisms, together with open issues and future directions.

Typology of Risks of Generative Text-to-Image Models

“…On the flip side, TTI systems can be used for malicious purposes. In the realm of misinformation and disinformation, players such as hyper-partisan media, authoritarian regimes, state disinformation actors, and cyber-criminals have been identified as potential malicious users [4,5,14]. "Information operations" [107] are broadly acknowledged as a malicious use case.…”

“…These risks are particularly acute for women and public figures, who face character assassination through fake news or deepfake pornographic content [57,106,121,172]. Moreover, the destabilising potential of generative AI, such as providing visual legitimacy to populist or nationalist conspiracies and fake news [5,29,100,171], should not be overlooked. It is crucial to recognise that while all media consumers are vulnerable to these harms, those with less societal power to contest falsehoods -people of colour, women, LGBTQ+ communities [121] -are particularly at risk.…”

“…The misuse of TTI models extends beyond sexualised imagery, leading to harmful likeness reproduction in various other forms. Examples include the creation of fake journalism profiles [89], or use in blackmail, revenge [71,116], or identity theft for scams [5,103]. Furthermore, TTI-enabled misinformation and disinformation can reinforce existing cognitive biases [4], amplifying narratives of "otherness" [61,153].…”

“…We wish to highlight the potential of TTI technologies to cause political harms. TTI systems can further damage political institutions and compromise the integrity of democratic discourse [29] through election interference [5,171], enabling misinformation and disinformation actors to operate at larger scales, and creating "evidence" to legitimize fake news or propaganda [107,112,171]. In addition we highlight the risks posed wherein TTI systems are used to generate culturally offensive content.…”

“…Furthermore, these risks are concerning for communities who are more susceptible to democratic and social instabilities and may have fewer data protections [82,96,171]. The detrimental effects of TTI-enabled misinformation and disinformation extend to financial markets and economies, with potential for disruption [5,100,120,130]. TTI systems also has the potential to increase the risk of conflict and state violence [27,113].…”