Multisemantic Path Neural Network for Deepfake Detection

Abstract

With the continuous development of deep learning techniques, it is now easy for anyone to swap faces in videos. Researchers find that the abuse of these techniques threatens cyberspace security; thus, face forgery detection is a popular research topic. However, current detection methods do not fully use the semantic features of deepfake videos. Most previous work has only divided the semantic features, the importance of which may be unequal, by experimental experience. To solve this problem, we propose a new framework, which is the multisemantic pathway network (MSPNN) for fake face detection. This method comprehensively captures forged information from the dimensions of microscopic, mesoscopic, and macroscopic features. These three kinds of semantic information are given learnable weights. The artifacts of deepfake images are more difficult to observe in a compressed video. Therefore, preprocessing is proposed to detect low-quality deepfake videos, including multiscale detail enhancement and channel information screening based on the compression principle. Center loss and cross-entropy loss are combined to further reduce intraclass spacing. Experimental results show that MSPNN is superior to contrast methods, especially low-quality deepfake video detection.

A survey on deepfake video detection datasets

“…In Table 3, we show a collection of articles (Reference) and their average scores according to the datasets. Faceforensics++ [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38] 94.2 [39], [40], [41], [42], [43] 93. 55 3 DeepFakeDetection [44] 90.80 --4 UADFV [45], [46], [47] 93.4 [48], [49] 98.7 5…”

“…Celeb-DF [25], [29], [33], [35], [36], [41], [45], [47], [52] 85.79 [27], [43], [49], [50], [53], [54] 82.23 7 Celeb-DFv2 [34] 99.31 [40], [32], [31] 88.01 8 DFDC preview [24], [45], [55], [56], [57] 91.61 [51] 84.4 9 DFDC [29], [35], [42], [58] 83.27 [29], [31], [41], [42], [59] 89. According to the previous study the manipulated video detection models are divided into two different forms, within-domain and cross-domain.…”