Self-portrait to ID Document face matching: CNN-Based face verification in cross-domain scenario
Resumo
Face verification approaches determine whether two given faces are from the same person. Recently, a new demand for face verification application which has become popular in commercial applications is the self-portrait and ID face matching, in which we compare the faces of a selfie shot by a subject and the face in a picture of her identification document. In this work, we proposed a novel approach for face verification in a cross-domain scenario, assuming we have only two images for each subject in the dataset. The method is based on siamese architecture with triplet-loss function. Experiments show the proposed model reaches good effectiveness for cross-domain face verification with low error rates, in comparison to other works of the literature.
Palavras-chave:
Selfie-document, Face Verification, Triplet-loss, Siamese networks
Referências
W. Zhao, R. Chellappa, J. Phillips, and A. Rosenfeld, “Face recognition: A literature survey,” ACM Computing Surveys, vol. 35, no. 4, pp. 399-458, 2003.
R. Chellappa, P. Sinha, and P. Phillips, “Face recognition by computers and humans,” IEEE Computer, vol. 43, no. 2, pp. 46-55, 2010.
R. Vareto, S. Silva, F. Costa, and W. R. Schwartz, “Face verification based on relational disparity features and partial least squares models,” in IEEE Conference on Graphics, Patterns and Images, 2017, pp. 209-215.
Y. Sun, X. Wang, and X. Tang, “Hybrid deep learning for face verification,” in IEEE International Conference on Computer Vision, 2013, pp. 1489-1496.
__, “Deep learning face representation by joint identificationverification,” arXiv preprint arXiv:1406.4773, 2014.
W. Wang, Y. Fu, X. Qian, Y.-G. Jiang, Q. Tian, and X. Xue, “Fm2unet: Face morphological multi-branch network for makeup-invariant face verification,” in IEEE Conference on Computer Vision and Pattern Recognition, 2020, pp. 5730-5740.
G. Folego, M. A. Angeloni, J. A. Stuchi, A. Godoy, and A. Rocka, “Cross-domain face verification: Matching id document and self-portrait photographs,” in XII Workshop de Visão Computacional, 2006, pp. 311-316.
R. Paliwal, S. Yadav, and N. Nain, “Faceid: Verification of face in selfie and id document,” in Springer International Conference on Computer Vision and Image Processing, 2019, pp. 443-454.
Y. Shi and A. K. Jain, “Docface+: Id document to selfie matching,” IEEE Transactions on Biometrics, Behavior, and Identity Science, vol. 1, no. 1, pp. 56-67, 2019.
J. Oliveira, G. Souza, A. Rocha, F. Deus, and A. Marana, “Crossdomain deep face matching for real banking security systems,” in IEEE International Conference on eDemocracy & eGovernment. IEEE, 2020, pp. 21-28.
F. Schroff, D. Kalenichenko, and J. Philbin, “Facenet: A unified embedding for face recognition and clustering,” in IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 815-823.
J. Lu, Y.-P. Tan, and G. Wang, “Discriminative multimanifold analysis for face recognition from a single training sample per person,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 1, pp. 39-51, 2013.
D. Yi, Z. Lei, and S. Z. Li, “Towards pose robust face recognition,” in 2013 IEEE Conference on Computer Vision and Pattern Recognition, 2013, pp. 3539-3545.
S. Zafeiriou, G. Tzimiropoulos, M. Petrou, and T. Stathaki, “Regularized kernel discriminant analysis with a robust kernel for face recognition and verification,” IEEE Transactions on Neural Networks and Learning Systems, vol. 23, no. 3, pp. 526-534, 2012.
P. Viola and M. Jones, “Rapid object detection using a boosted cascade of simple features,” in Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, vol. 1, 2001, pp. I-I.
P. Viola and M. J. Jones, “Robust real-time face detection,” International Journal of Computer Vision, vol. 57, no. 2, pp. 137-154, 2004.
T. Ahonen, A. Hadid, and M. Pietikainen, “Face description with local binary patterns: Application to face recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 12, pp. 2037-2041, 2006.
Z. M. Hafed and M. D. Levine, “Face recognition using the discrete cosine transform,” International Journal of Computer Vision, vol. 43, no. 3, pp. 167-188, 2001.
M. J. Er, W. Chen, and S. Wu, “High-speed face recognition based on discrete cosine transform and rbf neural networks,” IEEE Transactions on Neural Networks, vol. 16, no. 3, pp. 679-691, 2005.
G. B. Huang, H. Lee, and E. Learned-Miller, “Learning hierarchical representations for face verification with convolutional deep belief networks,” in 2012 IEEE Conference on Computer Vision and Pattern Recognition, 2012, pp. 2518-2525.
P. McCullagh and J. A. Nelder, Generalized linear models. Routledge, 2019.
C. Cortes and V. Vapnik, “Support-vector networks,” Machine Learning, vol. 20, no. 3, pp. 273-297, 1995.
W. Chen, M. J. Er, and S. Wu, “Illumination compensation and normalization for robust face recognition using discrete cosine transform in logarithm domain,” IEEE Transactions on Systems, Man, and Cybernetics, vol. 36, no. 2, pp. 458-466, 2006.
V. Starovoitov, D. Samal, and B. Sankur, “Matching of faces in camera images and document photographs,” in 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100), vol. 4, 2000, pp. 2349-2352.
O. Parkhi, A. Vedaldi, and A. Zisserman, “Deep face recognition,” in British Machine Vision Conference, 2015, pp. 1-12.
T. Bourlai, A. Ross, and A. Jain, “On matching digital face images against scanned passport photos,” in IEEE International Conference on Biometrics, Identity and Security, 2009, pp. 1-10.
Y. Shi and A. K. Jain, “Docface: Matching id document photos to selfies,” in 2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS), 2018, pp. 1-8.
V. Albiero, N. Srinivas, E. Villalobos, J. Perez-Facuse, R. Rosenthal, D. Mery, K. Ricanek, and K. W. Bowyer, “Identity document to selfie face matching across adolescence,” in IEEE International Joint Conference on Biometrics, 2019, pp. 1-9.
G. B. M. Mattar, T. Berg, and E. Learned-Miller, “Labeled faces in the wild: A database forstudying face recognition in unconstrained environments,” in Workshop on faces in Real-Life' Images: detection, alignment, and recognition, 2008, pp. 1-14.
K. Zhang, Z. Zhang, Z. Li, and Y. Qiao, “Joint face detection and alignment using multitask cascaded convolutional networks,” IEEE Signal Processing Letters, vol. 23, no. 10, pp. 1499-1503, 2016.
S. Pizer, E. Amburn, J. Austin, R. Cromartie, A. Geselowitz, T. Greer, B. Romeny, J. Zimmerman, and K. Zuiderveld, “Adaptive histogram equalization and its variations,” Elsevier Computer Vision, Graphics, and Image Processing, vol. 39, no. 3, pp. 355-368, 1987.
J. Bromley, I. Guyon, Y. LeCun, E. Säckinger, and R. Shah, “Signature verification using a “siamese” time delay neural network,” Advances in Neural Information Processing Systems, vol. 6, pp. 737-744, 1993.
Q. Guo, W. Feng, C. Zhou, R. Huang, L. Wan, and S. Wang, “Learning dynamic siamese network for visual object tracking,” in IEEE International Conference on Computer Vision, 2017, pp. 1763-1771.
C. Zhang, W. Liu, H. Ma, and H. Fu, “Siamese neural network based gait recognition for human identification,” in IEEE International Conference on Acoustics, Speech and Signal Processing, 2016, pp. 2832-2836.
Y. Taigman, M. Yang, M. Ranzato, and L. Wolf, “Deepface: Closing the gap to human-level performance in face verification,” in IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 1701-1708.
G. Koch, R. Zemel, and R. Salakhutdinovn, “Siamese neural networks for one-shot image recognition,” in Deep Learning Workshop at International Conference on Machine Learning, 2015.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770-778.
D. Kingma and J. Ba, “Adam: A method for stochastic optimization,” 2014.
T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, “Focal loss for dense object detection,” in IEEE International Conference on Computer Vision, 2017, pp. 2980-2988.
R. Chellappa, P. Sinha, and P. Phillips, “Face recognition by computers and humans,” IEEE Computer, vol. 43, no. 2, pp. 46-55, 2010.
R. Vareto, S. Silva, F. Costa, and W. R. Schwartz, “Face verification based on relational disparity features and partial least squares models,” in IEEE Conference on Graphics, Patterns and Images, 2017, pp. 209-215.
Y. Sun, X. Wang, and X. Tang, “Hybrid deep learning for face verification,” in IEEE International Conference on Computer Vision, 2013, pp. 1489-1496.
__, “Deep learning face representation by joint identificationverification,” arXiv preprint arXiv:1406.4773, 2014.
W. Wang, Y. Fu, X. Qian, Y.-G. Jiang, Q. Tian, and X. Xue, “Fm2unet: Face morphological multi-branch network for makeup-invariant face verification,” in IEEE Conference on Computer Vision and Pattern Recognition, 2020, pp. 5730-5740.
G. Folego, M. A. Angeloni, J. A. Stuchi, A. Godoy, and A. Rocka, “Cross-domain face verification: Matching id document and self-portrait photographs,” in XII Workshop de Visão Computacional, 2006, pp. 311-316.
R. Paliwal, S. Yadav, and N. Nain, “Faceid: Verification of face in selfie and id document,” in Springer International Conference on Computer Vision and Image Processing, 2019, pp. 443-454.
Y. Shi and A. K. Jain, “Docface+: Id document to selfie matching,” IEEE Transactions on Biometrics, Behavior, and Identity Science, vol. 1, no. 1, pp. 56-67, 2019.
J. Oliveira, G. Souza, A. Rocha, F. Deus, and A. Marana, “Crossdomain deep face matching for real banking security systems,” in IEEE International Conference on eDemocracy & eGovernment. IEEE, 2020, pp. 21-28.
F. Schroff, D. Kalenichenko, and J. Philbin, “Facenet: A unified embedding for face recognition and clustering,” in IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 815-823.
J. Lu, Y.-P. Tan, and G. Wang, “Discriminative multimanifold analysis for face recognition from a single training sample per person,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 1, pp. 39-51, 2013.
D. Yi, Z. Lei, and S. Z. Li, “Towards pose robust face recognition,” in 2013 IEEE Conference on Computer Vision and Pattern Recognition, 2013, pp. 3539-3545.
S. Zafeiriou, G. Tzimiropoulos, M. Petrou, and T. Stathaki, “Regularized kernel discriminant analysis with a robust kernel for face recognition and verification,” IEEE Transactions on Neural Networks and Learning Systems, vol. 23, no. 3, pp. 526-534, 2012.
P. Viola and M. Jones, “Rapid object detection using a boosted cascade of simple features,” in Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, vol. 1, 2001, pp. I-I.
P. Viola and M. J. Jones, “Robust real-time face detection,” International Journal of Computer Vision, vol. 57, no. 2, pp. 137-154, 2004.
T. Ahonen, A. Hadid, and M. Pietikainen, “Face description with local binary patterns: Application to face recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 12, pp. 2037-2041, 2006.
Z. M. Hafed and M. D. Levine, “Face recognition using the discrete cosine transform,” International Journal of Computer Vision, vol. 43, no. 3, pp. 167-188, 2001.
M. J. Er, W. Chen, and S. Wu, “High-speed face recognition based on discrete cosine transform and rbf neural networks,” IEEE Transactions on Neural Networks, vol. 16, no. 3, pp. 679-691, 2005.
G. B. Huang, H. Lee, and E. Learned-Miller, “Learning hierarchical representations for face verification with convolutional deep belief networks,” in 2012 IEEE Conference on Computer Vision and Pattern Recognition, 2012, pp. 2518-2525.
P. McCullagh and J. A. Nelder, Generalized linear models. Routledge, 2019.
C. Cortes and V. Vapnik, “Support-vector networks,” Machine Learning, vol. 20, no. 3, pp. 273-297, 1995.
W. Chen, M. J. Er, and S. Wu, “Illumination compensation and normalization for robust face recognition using discrete cosine transform in logarithm domain,” IEEE Transactions on Systems, Man, and Cybernetics, vol. 36, no. 2, pp. 458-466, 2006.
V. Starovoitov, D. Samal, and B. Sankur, “Matching of faces in camera images and document photographs,” in 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100), vol. 4, 2000, pp. 2349-2352.
O. Parkhi, A. Vedaldi, and A. Zisserman, “Deep face recognition,” in British Machine Vision Conference, 2015, pp. 1-12.
T. Bourlai, A. Ross, and A. Jain, “On matching digital face images against scanned passport photos,” in IEEE International Conference on Biometrics, Identity and Security, 2009, pp. 1-10.
Y. Shi and A. K. Jain, “Docface: Matching id document photos to selfies,” in 2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS), 2018, pp. 1-8.
V. Albiero, N. Srinivas, E. Villalobos, J. Perez-Facuse, R. Rosenthal, D. Mery, K. Ricanek, and K. W. Bowyer, “Identity document to selfie face matching across adolescence,” in IEEE International Joint Conference on Biometrics, 2019, pp. 1-9.
G. B. M. Mattar, T. Berg, and E. Learned-Miller, “Labeled faces in the wild: A database forstudying face recognition in unconstrained environments,” in Workshop on faces in Real-Life' Images: detection, alignment, and recognition, 2008, pp. 1-14.
K. Zhang, Z. Zhang, Z. Li, and Y. Qiao, “Joint face detection and alignment using multitask cascaded convolutional networks,” IEEE Signal Processing Letters, vol. 23, no. 10, pp. 1499-1503, 2016.
S. Pizer, E. Amburn, J. Austin, R. Cromartie, A. Geselowitz, T. Greer, B. Romeny, J. Zimmerman, and K. Zuiderveld, “Adaptive histogram equalization and its variations,” Elsevier Computer Vision, Graphics, and Image Processing, vol. 39, no. 3, pp. 355-368, 1987.
J. Bromley, I. Guyon, Y. LeCun, E. Säckinger, and R. Shah, “Signature verification using a “siamese” time delay neural network,” Advances in Neural Information Processing Systems, vol. 6, pp. 737-744, 1993.
Q. Guo, W. Feng, C. Zhou, R. Huang, L. Wan, and S. Wang, “Learning dynamic siamese network for visual object tracking,” in IEEE International Conference on Computer Vision, 2017, pp. 1763-1771.
C. Zhang, W. Liu, H. Ma, and H. Fu, “Siamese neural network based gait recognition for human identification,” in IEEE International Conference on Acoustics, Speech and Signal Processing, 2016, pp. 2832-2836.
Y. Taigman, M. Yang, M. Ranzato, and L. Wolf, “Deepface: Closing the gap to human-level performance in face verification,” in IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 1701-1708.
G. Koch, R. Zemel, and R. Salakhutdinovn, “Siamese neural networks for one-shot image recognition,” in Deep Learning Workshop at International Conference on Machine Learning, 2015.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770-778.
D. Kingma and J. Ba, “Adam: A method for stochastic optimization,” 2014.
T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, “Focal loss for dense object detection,” in IEEE International Conference on Computer Vision, 2017, pp. 2980-2988.
Publicado
22/11/2021
Como Citar
COSTA, Filipe; MELO, Marcos Vinícius L.; GADELHA, Igor; FÔLEGO, Guilherme; GAMBARO, Larissa; RODRIGUES, André.
Self-portrait to ID Document face matching: CNN-Based face verification in cross-domain scenario. In: WORKSHOP DE VISÃO COMPUTACIONAL (WVC), 17. , 2021, Online.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2021
.
p. 31-36.
DOI: https://doi.org/10.5753/wvc.2021.18885.
