Detecting and Simulating Artifacts in GAN Fake Images

Xu, Z.; Karaman, Svebor; Chang, Shih-Fu

doi:10.1109/wifs47025.2019.9035107

Cited by 371 publications

(273 citation statements)

References 20 publications

Supporting

Mentioning

246

Contrasting

Unclassified

Order By: Relevance

“…They described a new method based on monitoring neuron behaviors of a dedicated CNN to detect faces generated by Deepfake technologies. The comparison with Zhang et al [17] demonstrated an average detection accuracy of more than 90%…”

Section: B Deepfake Detection Methodsmentioning

confidence: 79%

“…This preliminary insight was detected by Guarnera et al [4] in which the authors tried to roughly detect Deepfakes by means of well-known forgery detection tools ( [2], [15], [16]) with only few insights for future works as results. The analysis in the Fourier domain was employed by Zhang et al [17] in a rather naive strategy which delivered in any case good performances. Later, an interesting work known as FakeSpotter was proposed by Wang et al [18].…”

Section: B Deepfake Detection Methodsmentioning

confidence: 99%

“…In this Section a detailed discussion is carried out, comparing results obtained by the proposed approach with the best literature methods: Wang et al [18] (the authors of FakeSpotter), Zhang et al [17] (the authors of AutoGAN) and Wang et al [19] were taken into account for comparisons in the Real Vs. Deepfake binary classification task.…”

Section: Comparisons With Deepfake Detection Methodsmentioning

confidence: 99%

“…However, another interesting insight comes from the rotation attacks: a rotation of 90 degrees anticlockwise, which is a rotation that does not introduce interpolation, unexpectedly FIGURE 9. Comparison of the proposed approach (Our) vs. FakeSpotter [18] and AutoGAN [17].…”

Section: Robustness Experimentsmentioning

confidence: 99%

See 3 more Smart Citations

Fighting Deepfake by Exposing the Convolutional Traces on Images

2020

View full text Add to dashboard Cite

show abstract

Section: B Deepfake Detection Methodsmentioning

confidence: 79%

Section: B Deepfake Detection Methodsmentioning

confidence: 99%

Section: Comparisons With Deepfake Detection Methodsmentioning

confidence: 99%

Section: Robustness Experimentsmentioning

confidence: 99%

See 2 more Smart Citations

Fighting Deepfake by Exposing the Convolutional Traces on Images

2020

View full text Add to dashboard Cite

show abstract

“…Although easily fooling the human, the state-of-the-art synthesized images can still be detected in many cases by current fake detection methods. The state-of-the-art synthesized methods often introduce artifact patterns into the image during generation, opening a chance for fake detectors [14,59]. Due to the current technical limitation, even worse, the image manipulation footprint will be inevitably left in a synthesized image, either by partial image manipulation [11,18,34] or full image synthesis [29][30][31].…”

Section: Introductionmentioning

confidence: 99%

FakePolisher: Making DeepFakes More Detection-Evasive by Shallow Reconstruction

Huang

Juefei-Xu

Wang

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

View full text Add to dashboard Cite

At this moment, GAN-based image generation methods are still imperfect, whose upsampling design has limitations in leaving some certain artifact patterns in the synthesized image. Such artifact patterns can be easily exploited (by recent methods) for difference detection of real and GAN-synthesized images. However, the existing detection methods put much emphasis on the artifact patterns, which can become futile if such artifact patterns were reduced. Towards reducing the artifacts in the synthesized images, in this paper, we devise a simple yet powerful approach termed FakePolisher that performs shallow reconstruction of fake images through a learned linear dictionary, intending to effectively and efficiently reduce the artifacts introduced during image synthesis. In particular, we first train a dictionary model to capture the patterns of real images. Based on this dictionary, we seek the representation of DeepFake images in a low dimensional subspace through linear projection or sparse coding. Then, we are able to perform shallow reconstruction of the 'fake-free' version of the DeepFake image, which largely reduces the artifact patterns DeepFake introduces. The comprehensive evaluation on 3 state-of-the-art DeepFake detection methods and fake images generated by 16 popular GAN-based fake image generation techniques, demonstrates the effectiveness of our technique. Overall, through reducing artifact patterns, our technique significantly reduces the accuracy of the 3 state-of-theart fake image detection methods, i.e., 47% on average and up to 93% in the worst case. Our results confirm the limitation of current fake detection methods and calls the attention of DeepFake researchers and practitioners for more general-purpose fake detection techniques. CCS CONCEPTS • Security and privacy → Human and societal aspects of security and privacy; • Computing methodologies → Computer vision.

show abstract

A projection‐domain correction method in CBCT reconstruction for head and neck radiotherapy using cycle‐GAN and nonlocal means filter

et al. 2023

View full text Add to dashboard Cite

BackgroundIn radiation treatments for head and neck tumors, cone‐beam computed tomography (CBCT) is employed for patient positioning and dose calculation of adaptive radiotherapy. However, the quality of CBCT is degraded by the scatter and noise, majorly impacting the accuracy of patient positioning and dose calculation.PurposeTo improve the quality of CBCT for patients with head and neck cancer, a projection‐domain CBCT correction method was proposed using a cycle‐consistent generative adversarial network (cycle‐GAN) and a nonlocal means filter (NLMF) based on a reference digitally reconstructed radiograph (DRR).MethodsA cycle‐GAN was initially trained to learn mapping from CBCT projections to a DRR using the data obtained from 30 patients. For each patient, 671 CBCT projections were measured for CBCT reconstruction. Moreover, 360 Digital Reconstructed Radiographs (DRR) were computed from each patient's planning computed tomography (CT), whose projection angles ranged from 0° to 359° with an interval of 1°. By applying the trained generator of the cycle‐GAN to the unseen CBCT projection, a synthetic DRR with considerably less scatter was obtained. However, annular artifacts were observed in the CBCT reconstructed with synthetic DRR. To address this issue, a NLMF based on reference DRR was used to further correct the synthetic DRR, which corrected the synthetic DRR using the calculated DRR as a reference image. Finally, the CBCT with no annular artifact and little noise was reconstructed with the corrected synthetic DRR. The proposed method was tested using the data of six patients. The corrected synthetic DRR and CBCT were compared with the corresponding real DRR and CT images. The structural preservation ability of the proposed method was evaluated using the Dice coefficients of the automatically extracted nasal cavity. Moreover, the image quality of CBCT corrected with the proposed method was objectively assessed with an five‐point human scoring system and compared with CT, original CBCT and CBCT corrected with other strategies.ResultsThe mean absolute value (MAE) of the relative error between the corrected synthetic and real DRR was <8%. The MAE between the corrected CBCT and corresponding CT was <30 HU. Moreover, the Dice coefficient of nasal cavity between the corrected CBCT image and the original image exceeded 98.8 for all the patients. Last but not least, the objective assessment of image quality showed the proposed method had an average score of 4.2 in overall image quality, which was higher than that of the original CBCT, CBCT reconstructed with synthetic DRR, and CBCT reconstructed with projections filtered with NLMF only.ConclusionsThe proposed method can considerably improve the CBCT image quality with little anatomical distortion, improving the accuracy of radiotherapy for head and neck patients.

show abstract

Detecting and Simulating Artifacts in GAN Fake Images

Cited by 371 publications

References 20 publications

Fighting Deepfake by Exposing the Convolutional Traces on Images

Fighting Deepfake by Exposing the Convolutional Traces on Images

FakePolisher: Making DeepFakes More Detection-Evasive by Shallow Reconstruction

A projection‐domain correction method in CBCT reconstruction for head and neck radiotherapy using cycle‐GAN and nonlocal means filter

Contact Info

Product

Resources

About