Advanced search
2 files | 759.93 KB Add to list

Improved deepfake video detection using convolutional vision transformer

Author
Organization
Project
Abstract
Deepfakes are hyper-realistic videos in which the faces are replaced, swapped, or forged using deep-learning models. This potent media manipulation techniques hold promise for applications across various domains. Yet, they also present a significant risk when employed for malicious intents like iden-tity fraud, phishing, spreading false information, and executing scams. In this work, we propose a novel and improved Deepfake video detector that uses a Convolutional Vision Transformer (CViT2), which builds on the concepts of our previous work (CViT). The CViT architecture consists of two components: a Convolutional Neural Network that extracts learnable features, and a Vision Transformer that categorizes these learned features using an attention mechanism. We trained and evaluted our model on 5 datasets, namely Deepfake Detection Challenge Dataset (DFDC), FaceForensics++ (FF++) I, Celeb-DF v2, Deep-fakeTIMIT, and TrustedMedia. On the test sets unseen during training, we achieved an accuracy of 95 %, 94.8 %, 98.3 % and 76.7% on the DFDC, FF++ , Celeb-DF v2, and TIMIT datasets, respectively. In conclusion, our proposed Deepfake detector can be used in the battle against misinformation and other forensic use cases.
Keywords
Deepfake Video Detection, Vision Transformer, Convolutional Neural Network, Misinformation Detection, Multimedia Forensics

Downloads

  • DS792 acc.pdf
    • full text (Accepted manuscript)
    • |
    • open access
    • |
    • PDF
    • |
    • 353.93 KB
  • (...).pdf
    • full text (Published version)
    • |
    • UGent only
    • |
    • PDF
    • |
    • 406.00 KB

Citation

Please use this url to cite or link to this publication:

MLA
Deressa, Deressa Wodajo, et al. “Improved Deepfake Video Detection Using Convolutional Vision Transformer.” 2024 IEEE GAMING, ENTERTAINMENT, AND MEDIA CONFERENCE, GEM 2024, IEEE, 2024, pp. 492–97, doi:10.1109/gem61861.2024.10585593.
APA
Deressa, D. W., Lambert, P., Van Wallendael, G., Atnafu, S., & Mareen, H. (2024). Improved deepfake video detection using convolutional vision transformer. 2024 IEEE GAMING, ENTERTAINMENT, AND MEDIA CONFERENCE, GEM 2024, 492–497. https://doi.org/10.1109/gem61861.2024.10585593
Chicago author-date
Deressa, Deressa Wodajo, Peter Lambert, Glenn Van Wallendael, Solomon Atnafu, and Hannes Mareen. 2024. “Improved Deepfake Video Detection Using Convolutional Vision Transformer.” In 2024 IEEE GAMING, ENTERTAINMENT, AND MEDIA CONFERENCE, GEM 2024, 492–97. IEEE. https://doi.org/10.1109/gem61861.2024.10585593.
Chicago author-date (all authors)
Deressa, Deressa Wodajo, Peter Lambert, Glenn Van Wallendael, Solomon Atnafu, and Hannes Mareen. 2024. “Improved Deepfake Video Detection Using Convolutional Vision Transformer.” In 2024 IEEE GAMING, ENTERTAINMENT, AND MEDIA CONFERENCE, GEM 2024, 492–497. IEEE. doi:10.1109/gem61861.2024.10585593.
Vancouver
1.
Deressa DW, Lambert P, Van Wallendael G, Atnafu S, Mareen H. Improved deepfake video detection using convolutional vision transformer. In: 2024 IEEE GAMING, ENTERTAINMENT, AND MEDIA CONFERENCE, GEM 2024. IEEE; 2024. p. 492–7.
IEEE
[1]
D. W. Deressa, P. Lambert, G. Van Wallendael, S. Atnafu, and H. Mareen, “Improved deepfake video detection using convolutional vision transformer,” in 2024 IEEE GAMING, ENTERTAINMENT, AND MEDIA CONFERENCE, GEM 2024, Turin, Italy, 2024, pp. 492–497.
@inproceedings{01J2X7QBWYDBDCG7RA4FYC35Z1,
  abstract     = {{Deepfakes are hyper-realistic videos in which the faces are replaced, swapped, or forged using deep-learning models. This potent media manipulation techniques hold promise for applications across various domains. Yet, they also present a significant risk when employed for malicious intents like iden-tity fraud, phishing, spreading false information, and executing scams. In this work, we propose a novel and improved Deepfake video detector that uses a Convolutional Vision Transformer (CViT2), which builds on the concepts of our previous work (CViT). The CViT architecture consists of two components: a Convolutional Neural Network that extracts learnable features, and a Vision Transformer that categorizes these learned features using an attention mechanism. We trained and evaluted our model on 5 datasets, namely Deepfake Detection Challenge Dataset (DFDC), FaceForensics++ (FF++) I, Celeb-DF v2, Deep-fakeTIMIT, and TrustedMedia. On the test sets unseen during training, we achieved an accuracy of 95 %, 94.8 %, 98.3 % and 76.7% on the DFDC, FF++ , Celeb-DF v2, and TIMIT datasets, respectively. In conclusion, our proposed Deepfake detector can be used in the battle against misinformation and other forensic use cases.}},
  author       = {{Deressa, Deressa Wodajo and Lambert, Peter and Van Wallendael, Glenn and Atnafu, Solomon and Mareen, Hannes}},
  booktitle    = {{2024 IEEE GAMING, ENTERTAINMENT, AND MEDIA CONFERENCE, GEM 2024}},
  isbn         = {{9798350374537}},
  issn         = {{2831-5510}},
  keywords     = {{Deepfake Video Detection,Vision Transformer,Convolutional Neural Network,Misinformation Detection,Multimedia Forensics}},
  language     = {{eng}},
  location     = {{Turin, Italy}},
  pages        = {{492--497}},
  publisher    = {{IEEE}},
  title        = {{Improved deepfake video detection using convolutional vision transformer}},
  url          = {{http://doi.org/10.1109/gem61861.2024.10585593}},
  year         = {{2024}},
}

Altmetric
View in Altmetric
Web of Science
Times cited: