|Journal of Information Hiding and Privacy Protection
A Survey of Anti-forensic for Face Image Forgery
Engineering Research Center of Digital Forensics of Ministry of Education, School of Computer, Nanjing University of Information Science & Technology, Nanjing, 210044, China
*Corresponding Author: Haitao Zhang. Email: firstname.lastname@example.org
Received: 25 April 2022; Accepted: 28 May 2022
Abstract: Deep learning related technologies, especially generative adversarial network, are widely used in the fields of face image tampering and forgery. Forensics researchers have proposed a variety of passive forensic and related anti-forensic methods for image tampering and forgery, especially face images, but there is still a lack of overview of anti-forensic methods at this stage. Therefore, this paper will systematically discuss the anti-forensic methods for face image tampering and forgery. Firstly, this paper expounds the relevant background, including the relevant tampering and forgery methods and forensic schemes of face images. The former mainly includes four aspects: conventional processing, fake face generation, face editing and face swapping; The latter is mainly the relevant forensic means based on spatial domain and frequency domain using deep learning technology. Then, this paper divides the existing anti-forensic works into three categories according to their method characteristics, namely hiding operation traces, forgery reconstruction and adversarial attack. Finally, this paper summarizes the limitations and prospects of the existing anti-forensic technologies.
Keywords: Anti-forensics; face tempering and forgery; forensics; generative adversarial network
Face images are widely used in daily life and work. However, with the rapid development of image processing technologies and the success of deep learning related technologies, people can not only edit and tamper with the existing face images, but also swap faces and reenact faces, and even generate fake faces that do not exist at all. The boundary between these forged and tampered face images and real face images becomes blurred, which is difficult to distinguish by human eyes. In addition, the software package provided free of charge on the Internet allows any individual to easily process or generate very realistic fake face images even without special skills. On the one hand, it opens the door to a series of amazing applications in different fields such as advertising and film production. On the other hand, in today’s world, once fake face images are maliciously used, they will pose a huge security threat and seriously affect social stability. For example, using fake face images to register accounts or swap faces will lead to privacy or security problems.
The challenges posed by fake face images have promoted the development of forensics technology [1,2], and researchers have proposed a large number of forensic solutions to address the above problems. The source of the tampered images is traced through active forensics, and the authenticity of the image is checked through passive forensics. Furthermore, an initial defense  is proposed to prevent forgers from successfully tampering with real faces. At present, passive forensics schemes have achieved very impressive performance in detecting image authenticity and are trusted by researchers. However, these forensic solutions may still have some kind of loopholes, and these loopholes may be utilized purposefully to make the tampered faces escape detection.
The research on anti-forensics [4,5] has emerged to reveal the vulnerabilities of existing forensic detectors. The so-called anti-forensics means that the tampered and forged image cannot be detected by image forensics technology after some processing, which reduces the performance of the detectors. Studying anti-forensics techniques in turn motivates researchers to come up with more reliable and robust forensic schemes, which are crucial in security-related forensics tasks. At present, there is still a lack of relevant research reviews on anti-forensic technologies. This article will briefly introduce relevant face tempering and forgery technology, forensics methods and systematically discuss the existing anti-forensic methods of images, especially face images.
2 Face Forgery and its Forensics
In this section, the relevant background of anti-forensic technology, including the main means of forgery and forensics schemes are discussed based on face images.
2.1 Face Forgery
In recent years, with the great success of deep learning technology, especially convolutional neural network (CNN) and generative adversarial network (GAN) , fake faces have been more and more difficult to be identified by human eyes. In this paper, according to the different ways of face tampering or generation, as shown in Fig. 1, face forgery technologies can be divided into four categories: conventional processing, face generation, face editing and face swapping.
Conventional processing. It mainly refers to the conventional image processing operations, mainly including resampling, JPEG compression, media filtering, contrast enhancement and image sharpening. These operations are very common and can be easily realized by image processing software such as Photoshop, GIMP and ACD See in real life.
Fake face generating. This forgery method mainly depends on the development of GAN technology. Through the adversarial training between the generator and the discriminator, the fake face images generated by the generator are closer and closer to the natural images in the process of weight alternating optimization.
The original GAN can only generate low resolution pictures, and the effect is relatively poor. Then, researchers carry out a series of research on the quality of face image generation. Radford et al.  combine convolution structure with GAN called DCGAN, which makes GAN easier to deal with many problems in the image field. Karras et al.  propose ProGAN based on step-by-step amplification training method to solve the problem of model collapse when generating high-resolution pictures. Then, inspired by style transfer, they decouple the input vectors to realize style control and propose StyleGAN , so that the generator can generate high quality and diverse face images with specified features. For the water droplet artifacts of faces generated by StyleGAN, they also propose StyleGAN2  to further improve the quality of the generated face. Lately, Karras et al.  improve the generator network structure of StyleGAN2 to make it have high-quality equivariance. Fig. 2 shows some examples of the above GAN generated face and it can be found that it is difficult for human eyes to detect the difference between them and natural faces in addition to DCGAN.
Face editing. Editing is often based on face generation, which can provide free attribute transformation of face, such as changing hair color, face skin color and so on. He et al.  propose a GAN structure called AttGAN, which applies attribute classification constraints to the generated image to ensure the correct changes of the required attributes. Choi et al.  use only one generative network called StarGAN to deal with the problem of generating images between multiple domains, and realizes further face attribute migration. Liu et al.  utilize differential attributes to further realize the editing of high-precision attributes of faces and propose STGAN.
Face swapping. This is to generate an image that retains the original face attributes, such as expression, illumination, angle, etc., and possess the appearance of the target face at the same time. There are Face2Face and FaceSwap based on computer graphics method, DeepFakes and NeuralTextuers  based on deep learning method.
2.2 Fake Face Forensics
Fake face image forensics aims to identify whether the measured image is real or fake, which can be regarded as a binary classification problem. The detection technology mainly utilizes the distinguishable features of natural faces and fake faces extracted by artificial design or neural network learning and the features are input into the classifier to realize classification.
According to different feature extraction methods, detection technology can be divided into two categories: artificial features based and deep learning based. The former is mainly based on the specific traces left by some forgery methods, such as capturing color component difference  or abnormal tooth details . It is lack of universality and is vulnerable to the changes of forgery methods and data distribution. The latter has gradually become the mainstream method because it can deal with more complex forgery methods. According to the difference of feature extraction and perspective, the scheme based on deep learning can be simply divided into: spatial domain-based methods and frequency domain-based methods.
Spatial domain-based methods. Some classical network structures, such as DenseNet , ResNet  and EfficientNet , have been directly applied to fake face detection and achieved good detection results. They directly extract feature information from the spatial domain of the image. In order to further improve the learning ability of neural network, researchers have introduced preprocessing, network module design, attention mechanism and other means. Both Nataraj et al.  and Goebel et al.  extract the co-occurrence matrix of three channels of images and utilize such features for classification. Guo et al.  suppress the interference of image content information and enhance tampering traces through adaptive extraction of high-frequency residuals. Liu et al.  introduce the gram-block to add the global texture information to different feature levels to improve the generalization and robust ability. Chen et al.  enhance the detection ability through the ensemble of luminance and chrominance component. Zhao et al.  utilize multiple attention mechanisms for deepfake detection. In addition, data enhancement strategy  can also work well.
Frequency domain-based methods. The forgery traces of fake face images also show abnormal characteristics in the frequency domain. Frank et al.  find that different up-sampling processes will lead to abnormal high-frequency components in the frequency domain and propose a detection method based on DCT transform. Similarly, Agarwal et al.  utilize color channel spectrum and capsule network. Qian et al.  propose F3-Net and carefully design the frequency domain information extraction module to enlarge the artifacts caused by the forgery process in the frequency domain. Chen et al.  propose MPSM method and improve frequency domain feature extraction of F3-Net. Liu et al.  propose a spatial shallow learning method SPSL based on phase spectrum information extraction.
Recently, the fake face detection models mentioned above have achieved considerable performance on many benchmark datasets, and even the detection accuracy has reached 100%.
3 Anti-forensic Methods
Face forgery anti-forensics, as the name suggests, is the confrontation form of forensic technology. Specifically, through some anti-forensic algorithm, the forged face cannot be detected by the forensics detector, i.e., the methods in Subsection 2.2 is invalid to detect the fake face in Subsection 2.1 after anti-forensic operations. The anti-forensic researches can be used to reveal the vulnerability of existing forensic methods and further promote the optimization and development of forensic algorithms. With the development of GANs and the emergency of adversarial attack, anti-forensic technology has ushered in new development opportunities and is easier to realize.
According to the different needs and means of anti-forensics, anti-forensic algorithms are classified as hiding operation traces, forgery reconstruction and adversarial attack.
3.1 Hiding Operation Traces
This kind of anti-forensic research has a long history and most of such anti-forensic methods are aimed at the operations in the conventional processing mentioned in the Subsection 2.1. Of course, there are also few relevant studies on the generated fake face.
For the JPEG compression, the anti-forensics has attracted extensive attention. The anti-forensics mainly focused on how to hide the quantization trace and block trace in the process of JPEG compression, which is realized by adding median filter and Gaussian white noise  or redistributing DCT coefficients by adding noise . For the contrast enhancement, the aim of anti-forensics is to make the image have no spike or gap effect on its histogram. The main methods include adjusting the image histogram by resampling or noise  and offsetting the original traces by forging traces . For the media filtering, it mainly attempts to use the optimization method to eliminate the left traces  and change the image pixel distribution by adding noise  to achieve the anti-forensic effect. For other forgery and tempering methods, Kirchner et al.  propose a method to hide image resampling traces. Lu et al.  mask the edge overshoot effect and end mutation trace caused by sharpening via adding jitter noise. In addition, for GAN-generated fake faces, Yu et al.  analyze the existence of “GAN fingerprints”. Therefore, trying to remove the “GAN fingerprint”  left by the generator of GAN can play a certain anti-forensics effect.
3.2 Forgery Reconstruction
Such anti-forensic methods mainly utilize the GAN technology and the study has been largely converted to image conversion problem and achieved good results. Specifically, by reconstructing the image to remove the relevant tampering and forgery features, the effect of fooling the detector can be achieved. This kind of anti-forensic means can be further divided into single forgery processing and multiple forgery processing. Specifically, the former mainly aims at a single way of tampering or forgery, the latter attempts to remove the traces of multiple ways of forgery at the same time and needs to ensure the anti-forensic effect under a single forgery way. Therefore, how to deal with the traces left by various tampering or forgery methods is still a great challenge. The following describes these two types of related articles in detail.
Single forgery processing. First, the related anti-forensic schemes for JPEG compression are introduced. Luo et al.  have achieved good results by using discriminator to capture statistical differences between images and uncompressed images. Wu et al.  use the loss function based on the high-frequency DCT coefficient to reconstruct the high-frequency component of the image. Wu et al.  propose JPEG restoration and anti-forensics GAN (JRA-GAN) and improve the loss function of high-frequency DCT coefficient in work .
Then, the anti-forensic algorithms for other conventional operations will be described as follows. Shen et al.  realize the similar processing of USM sharpening algorithm through pix2pix , so that the generated image has the characteristics of sharpening, but it cannot be simply regarded as a sharpened image, because the traditional sharpening operation is not used in the process. As a result, it can deceive CNN sharpening detector with high accuracy and achieve a good anti forensics effect. Kim et al.  reconstruct the image before median filtering, so that the reconstructed image can follow the statistical characteristics of the original image. Xie et al.  propose a dual domain anti-forensic GAN network, which discriminates in the forensics feature domain and spatial domain respectively through two discriminators, and achieve good anti-forensic effect.
Finally, for the computer-generated faces, researchers have achieved good anti-forensic effect by increasing the naturalness of the images. Concretely, Nguyen et al.  propose a generator called H-Net to transform the distribution feature of generated faces into the latent feature of natural faces. Aiming at the problem of insufficient color of face image in work , Peng et al.  propose a new GAN structure called CGR-GAN to convert style of natural face into generated faces inspired by style transfer. For the face Swapping, Ding et al.  propose a new GAN structure with multiple generators and multiple discriminators to resist DeepFake detection
Multiple forgery processing. The relevant anti-forensic work is relatively few. Wu et al.  design an anti-forensic algorithm for multiple editing processing by reconstructing images according to WGAN-GP network architecture. Based on this, they  extend the training mode and integrated mode, and introduce integrated multiple operation anti-forensic strategies.
3.3 Adversarial Attack
Firstly, the related terms and definitions related to this method are introduced.
Adversarial attack. Szegedy et al.  first propose the concept of adversarial example, that is, deliberately adding some well-designed perturbations to the input sample that people can’t detect, resulting in the model giving a wrong output with high confidence. The process is called adversarial attack.
White-box setting. The specific information of the model can be obtained when adversarial attacking, including network structure, network parameters and training data.
Black-box setting. The specific information of the model cannot be obtained.
Transferability. It can be understood as generalization. The adversarial example generated by attacking a white-box model also has an attack effect on the unknown model. This characteristic is called the transferability of adversarial examples.
The anti-forensic process realized by adversarial attack is shown in the Fig. 3. By adding adversarial perturbations to the fake face images, the forensics detector fails to detect the fake face.
The common adversarial attack baseline methods applied to anti-forensic tasks are introduced first. Fast gradient sign method (FGSM) , the most commonly used baseline anti-forensic attack method, is proposed by Goodfellow. The idea of FGSM algorithm is that an adversarial example is generated by performing a one-step gradient update along the direction of the gradient sign at each pixel. Based on FGSM attack, a series of iterative attack algorithms are derived. In the field of anti-forensics, project gradient descent (PGD)  and MIM  algorithms are the most representative. The former carries out multi-step perturbation along the increasing direction of the gradient through iteration, and recalculates the gradient direction after each small step. The latter introduces momentum on the basis of iteration.
Goebel et al.  try to make the co-occurrence matrix of the generated anti-forensic image close to the co-occurrence matrix of the target real image and achieve excellent results for the forensic detector based on co-occurrence matrix but it has poor transferability for other detectors. Wang et al.  apply FGSM attack and MIM attack to GAN-generated face images and find that most of the perturbations are concentrated in the Y channel, which seriously affect the visual quality, so they try to add perturbation restrictions to allocate more perturbations Under the Cb and Cr channels. Zhao et al.  introduce an ensemble strategy to improve the transferability of anti-forensics and apply the algorithm to GAN-generated face scenes . Li et al.  directly start from the source of the StyleGAN-generated facevy attacking the input latent vector and noise via PGD method to generate the anti-forensic faces. Jia et al.  propose a frequency domain adversarial attack against Deepfake detection instead of injecting adversarial perturbations into spatial domain and enhance the transferability. Hussain et al.  combine with PGD algorithm and some transformations, such as gaussian blur and translation, to realize the attack on DeepFake detectors and make adversarial perturbations robust.
4 Challenges and Prospects
At present, the forensics and anti-forensics related technologies of face tampering and forgery are still in their infancy, and this is a long-term and continuous process. The existing anti-forensic technologies still have some limitations.
The tradeoff between visual quality and anti-forensic effect. Due to the particularity of face image and the demand of anti-forensics, anti-forensic face needs to fool more detectors as many as possible without affecting the visual effect, that is, improve the transferability as much as possible. This balance is still a research focus and difficulty.
Insufficient interpretability. On the one hand, most of the existing advanced passive forensics technologies for face tampering and forgery are based on convolutional neural network, which leads to the lack of interpretability of the forensics detectors themselves. On the other hand, the anti-forensic method based on adversarial attack is more aimed at the loopholes of convolutional neural network, which lacks certain interpretability. In addition, the anti-forensic method based on forgery reconstruction also uses GAN technology based on deep learning, which also introduces new reconstruction traces.
Requirements of real scenes. The existing anti-forensic methods seldom consider various attacks in real scenes. For example, the anti-forensic means based on adversarial attack can easily make the adversarial perturbations that achieves the anti-forensic effect invalid when encountering operations such as blur and denoising. Therefore, the robustness of anti-forensic technology needs to be enhanced.
In the future, both forensic and anti-forensics methods need to make continuous progress to deal with the continuous development of face tampering and forgery technology. In view of the above limitations, the research on anti-forensics should dig more theoretical support and the trade-offs of generalization, robustness and visual effect on the basis of existing work.
This paper combs and summarizes the existing image anti-forensic technologies, especially the technologies related to face tampering and forgery. The objects, objectives, methods and limitations of existing anti-forensic schemes are systematically summarized. Aiming at face tampering and forgery, the existing anti-forensic methods reveal the relevant vulnerabilities of forensics detector by hiding operation traces, forgery reconstruction and adversarial attack.
In the future, face tampering and forgery anti-forensic technology will continue to tap the loopholes of existing forensic technology to develop more powerful forensic detectors to deal with the update iteration of tampering and forgery technology. Forensics and anti-forensic technologies have been continuously developed and improved in confrontation.
Funding Statement: The authors received no specific funding for this study.
Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.
|This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.