Quantitative Analysis of Visual XAI for Multiclass DeepFake Detection Based on CNN

• Leandro Santiago de Araújo UFF
• Flávio Luiz Seixas UFF
• Adriano Lima e Souza UFF
• Laura Tissi Tracierra Rezende UFF
• Hugo Coutinho Dutra UFF
• Miguel D. S. Wanderley Instituto de Ciência e Tecnologia Itaú
• Italo Antonio Duarte Oliveira Instituto de Ciência e Tecnologia Itaú

Resumo

The proliferation of deepfakes, highly realistic synthetic media generated through techniques such as Generative Adversarial Networks (GANs) and Diffusion Models, has introduced new challenges to digital media integrity and public trust. These manipulations are often indistinguishable to the human eye and pose significant threats, including misinformation, identity fraud, and the erosion of credibility in audiovisual content. Convolutional Neural Networks (CNNs) have demonstrated strong performance in detecting deepfakes by capturing subtle generation artifacts. However, their opaque decision-making processes raise concerns about interpretability and trustworthiness, particularly in sensitive domains such as media forensics and biometric authentication. This study conducts a quantitative analysis of visual explainable AI (XAI) techniques applied to CNN-based classifiers. Our approach employs a quantitative framework to evaluate the outputs produced by various visual XAI methods, including Grad-CAM, SmoothGrad, IntegratedGradients, InputXGradient, and Saliency map. The evaluation is conducted across multiple dimensions: Saliency Entropy, Salience-assessed Reaction to Noise, Salience Resilience to Geometrical Transformations, and Accuracy over SLIC-based Segmentation. Experiments are conducted on the FaceForensics++ dataset, which includes multiple manipulation types (e.g., FaceShifter, DeepFake, Face2Face, FaceSwap, and NeuralTextures). The quantitative analysis explores whether patterns in saliency behavior differ significantly between authentic and synthetic samples and whether these metrics can serve as auxiliary indicators for deepfake detection. This work contributes to enhancing the interpretability, reliability, and adoption of deep learning models in forensic applications.

Palavras-chave: Measurement, Deepfakes, Visualization, Accuracy, Statistical analysis, Forensics, Detectors, Feature extraction, Convolutional neural networks, Faces