An Unbiased Benchmark for Domain Generalization Face Anti-Spoofing - Almeida, Raul; Kamarowski, Bruno; Biesseck, Bernardo; Coelho, Luiz; Granada, Roger; Menotti, David (Anais Estendidos da Conference on Graphics, Patterns and Images (SIBGRAPI); 2025: Anais Estendidos da XXXVIII Conference on Graphics, Patterns and ImagesDO - 10.5753/sibgrapi.est.2025.38278, 2025)
Domain Generalization for Face Anti-Spoofing (DGFAS) is an area of growing interest due to its importance in fraud prevention in Face Recognition Systems. Current benchmarks used for DG-FAS in evaluating state-of-the-art methods allow verification of test set performance during training, which causes bias towards test data. Consequently, practitioners cannot properly translate research conclusions to real-world applications, since there is no access to labels for production data in practice. We propose as an alternative an unbiased benchmark where a validation dataset is used so that the model’s generalization capability is evaluated without compromising restricted data and the scientific rigor of research. Our experiments show that model performance benefits from current biased benchmarks and that introducing a new validation dataset makes for more challenging and scientifically rigorous benchmarks that also better represent real-world performance. We additionally experiment with training on current standard benchmarks and testing on WFAS, a recent in-the-wild large FAS dataset with more attack types than the standard datasets for DG-FAS, and similarly observe poor generalization capabilities for state-of-the-art models.