ESRL: efficient similarity representation learning for deepfake detection

For Deepfake detection, many existing works use the cross-entropy loss to enforce the classifier network to learn the mapping relationship from the RGB domain to the class domain, lacking an explicit constraint to guide the feature extraction network to learn discriminative features from an input im...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Multimedia tools and applications 2024-02, Vol.83 (31), p.76991-77007
Hauptverfasser: Wang, Feng, Zhang, Dengyong, Guo, Zhiqing, Wang, Dewang, Yang, Gaobo
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:For Deepfake detection, many existing works use the cross-entropy loss to enforce the classifier network to learn the mapping relationship from the RGB domain to the class domain, lacking an explicit constraint to guide the feature extraction network to learn discriminative features from an input image. This constrains the feature representation capability to expose deepfake. In this work, we analyze the feature extraction network in terms of both difference and similarity capabilities and propose a new constraint called similarity loss (SL) to improve the detection performance of the convolutional neural network (CNN) based detector. Moreover, according to the experimental results of the SL on data augmentation effectiveness, we propose a simple yet efficient framework, which is called as efficient similarity representation learning (ESRL), for deepfake detection. Extensive experiments on three public datasets (namely FF++, DFDC, and Celeb-DF) show that the feature extraction network trained with the help of SL can map forged faces and real faces to different feature embedding and map the same type of forged faces to similar feature embedding.
ISSN:1573-7721
1380-7501
1573-7721
DOI:10.1007/s11042-024-18447-x