Deepfakes generation and detection: state-of-the-art, open challenges, countermeasures, and way forward

Masood, Momina; Nawaz, Marriam; Malik, Khalid Mahmood; Javed, Ali; Irtaza, Aun; Malik, Hafiz

doi:10.1007/s10489-022-03766-z

Cited by 155 publications

(76 citation statements)

References 245 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Note that Similar to the works of DeepFake [dali18, MNM*21] or Bansal et al . [BMRS18], our work requires a training procedure for each video pair.…”

Section: Introductionmentioning

confidence: 56%

See 1 more Smart Citation

JOKR: Joint Keypoint Representation for Unsupervised Video Retargeting

Mokady

Tzaban

Benaim

et al. 2022

Computer Graphics Forum

View full text Add to dashboard Cite

In unsupervised video retargeting, content is transferred from one video to another while preserving the original appearance and style, without any additional annotations. While this challenge has seen substantial advancements through the use of deep neural networks, current methods struggle when the source and target videos are of shapes that are different in limb lengths or other body proportions. In this work, we consider this task for the case of objects of different shapes and appearances, that consist of similar skeleton connectivity and depict similar motion. We introduce JOKR-a JOint Keypoint Representation that captures the geometry common to both videos, while being disentangled from their unique styles. Our model first extracts unsupervised keypoints from the given videos. From this representation, two decoders reconstruct geometry and appearance, one for each of the input sequences. By employing an affine-invariant domain confusion term over the keypoints bottleneck, we enforce the unsupervised keypoint representations of both videos to be indistinguishable. This encourages the aforementioned disentanglement between motion and appearance, mapping similar poses from both domains to the same representation. This allows yielding a sequence with the appearance and style of one video, but the content of the other. Our applicability is demonstrated through challenging video pairs compared to state-of-the-art methods. Furthermore, we demonstrate that this geometry-driven representation enables intuitive control, such as temporal coherence and manual pose editing. Videos can be viewed in the supplement HTML.

show abstract

“…Note that Similar to the works of DeepFake [dali18, MNM*21] or Bansal et al . [BMRS18], our work requires a training procedure for each video pair.…”

Section: Introductionmentioning

confidence: 56%

“…[ZSBL19b] use facial landmarks to transfer face movement. DeepFake methods [dali18, MNM*21] were the first to swap faces between two input videos using deep neural networks. These methods either implicitly use 3D facial representations, or use facial landmarks to crop and align the face, and are again limited to facial data.…”

Section: Related Workmentioning

confidence: 99%

JOKR: Joint Keypoint Representation for Unsupervised Video Retargeting

Mokady

Tzaban

Benaim

et al. 2022

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…In the last few years, they have captured widespread attention for their uses in celebrity pornographic videos, revenge porn and fake news. Due to increased interests from amateurs, commercial companies and even governments, along with video generation easiness and improved video quality, they elicited responses from both industry and government to detect and limit their use as well as from scholars to understand their potential impact (Fallis, 2020;Vaccari and Chadwick, 2020;Masood et al, 2021). Still, research is limited and policies are lacking.…”

Section: Box 4 -The Rise Of Deepfakesmentioning

confidence: 99%

“…Raising awareness among the population and systems of prevention, detection and control could eventually result sufficient to tame the threats of deepfakes (Masood et al, 2021). As Vaccari and Chadwick (2020) argue, it is also possible that the reduction of trust in news on social media resulting from the uncertainty induced by deepfakes may not generate cynicism and alienation, but skepticism.…”

Section: Box 4 -The Rise Of Deepfakesmentioning

confidence: 99%

The Algorithmic Public Opinion: a Policy Overview

Reviglio¹

2022

Preprint

View full text Add to dashboard Cite

The algocount project seeks to contribute to concerns on the political and policy implications of the rise of algorithms. The diffusion of an “algorithmic public opinion” specifically regards the concern that algorithmic systems disrupt previous patterns of individual and public opinion formation. Few analyses explored this phenomenon throughout a multidisciplinary comprehensive policy lens. The algocount project seeks to address this gap by discussing innovative policy approaches and proposals to the challenges of the current algorithmic public opinion. Firstly, I introduce the main algorithms, concerns and actors involved in the algorithmic public opinion. Then, I discuss the main policy approaches to the systemic challenges of the most relevant algorithmic systems: recommender systems, disinformation, political microtargeting, content moderation and social bots. The goal of this report is to provide a critical, comprehensive and longterm oriented policy overview of the algorithmic public opinion.

show abstract

“…It is worth noting that technology likely has an important role in identifying and removing fakes from online epistemic environments(Alonso et al, 2021;Khan et al, 2021;Masood et al, 2021).…”

mentioning

confidence: 99%

Real Fakes: The Epistemology of Online Misinformation

Harris

2022

Philos. Technol.

View full text Add to dashboard Cite

Many of our beliefs are acquired online. Online epistemic environments are replete with fake news, fake science, fake photographs and videos, and fake people in the form of trolls and social bots. The purpose of this paper is to investigate the threat that such online fakes pose to the acquisition of knowledge. I argue that fakes can interfere with one or more of the truth, belief, and warrant conditions on knowledge. I devote most of my attention to the effects of online fakes on satisfaction of the warrant condition, as these have received comparatively little attention. I consider three accounts of the conditions under which fakes compromise the warrant condition. I argue for the third of these accounts, according to which the propensity of fakes to exist in an environment threatens warrant acquisition in that environment. Finally, I consider some limitations on the epistemic threat of fakes and suggest some strategies by which this threat can be mitigated.

show abstract

Deepfakes generation and detection: state-of-the-art, open challenges, countermeasures, and way forward

Cited by 155 publications

References 245 publications

JOKR: Joint Keypoint Representation for Unsupervised Video Retargeting

JOKR: Joint Keypoint Representation for Unsupervised Video Retargeting

The Algorithmic Public Opinion: a Policy Overview

Real Fakes: The Epistemology of Online Misinformation

Contact Info

Product

Resources

About