Eye movements and human face perception: An holistic analysis and proficiency classification based on frontal 2D face images

  • Victor P. L. Varela FEI
  • Estela Ribeiro FEI
  • Pedro A. S. S. Orona FEI
  • Carlos E. Thomaz FEI


Human faces convey a collection of information, such as gender, identity, and emotional states. Therefore, understanding the differences between volunteers’ eye movements on benchmark tests of face recognition and perception can explicitly indicate the most discriminating regions to improve performance in this visual cognitive task. The aim of this work is to qualify and classify these eye strategies using multivariate statistics and machine learning techniques, achieving up to 94.8% accuracy. Our experimental results show that volunteers have focused their visual attention, on average, at the eyes, but those with superior performance in the tests carried out have looked at the nose region more closely.


Bate, S. and Cook, S. J. (2012). Covert recognition relies on affective valence in developmental prosopagnosia: Evidence from the skin conductance response. Neuropsychology, 26(5):670.

Bate, S., Parris, B., Haslam, C., and Kay, J. (2010). Socio-emotional functioning and face recognition ability in the normal population. Personality and Individual Differences, 48(2):239–242.

Bengio, Y. and Grandvalet, Y. (2004). No unbiased estimator of the variance of k-fold cross-validation. Journal of machine learning research, 5(Sep):1089–1105.

Bobak, A. K. (2016). Theoretical and real-world applications of superior face recognition. PhD thesis, Bournemouth University.

Bodamer, J. (1947). Die prosop-agnosie. Archiv für Psychiatrie und Nervenkrankheiten, 179(1-2):6–53.

Bombari, D., Schmid, P. C., Schmid Mast, M., Birri, S., Mast, F. W., and Lobmaier, J. S. (2013). Emotion recognition: The role of featural and configural face information. The Quarterly Journal of Experimental Psychology, 66(12):2426–2442.

Bruce, V. and Young, A. W. (2012). Face perception. Psychology Press.

Burton, A. M., White, D., and McNeill, A. (2010). The glasgow face matching test. Behavior Research Methods, 42(1):286–291.

Caldara, R., Schyns, P., Mayer, E., Smith, M. L., Gosselin, F., and Rossion, B. (2005). Does prosopagnosia take the eyes out of face representations? evidence for a defect in representing diagnostic facial information following brain damage. Journal of cognitive neuroscience, 17(10):1652–1666.

Chan, C. Y., Chan, A. B., Lee, T. M., and Hsiao, J. H. (2018). Eye-movement patterns in face recognition are associated with cognitive decline in older adults. Psychonomic bulletin & review, pages 1–8.

Duchaine, B. and Nakayama, K. (2006). The cambridge face memory test: Results for neurologically intact individuals and an investigation of its validity using inverted face stimuli and prosopagnosic participants. Neuropsychologia, 44(4):576–585.

Gaur, R. P. and Jariwala, K. N. (2014). A survey on methods and models of eye tracking, head pose and gaze estimation. In Journal of Emerging Technologies and Innovative Research, volume 1. JETIR.

Gobbini, M. I. and Haxby, J. V. (2007). Neural systems for recognition of familiar faces. Neuropsychologia, 45(1):32–41.

Grüter, T., Grüter, M., and Carbon, C.-C. (2008). Neural and genetic foundations of face recognition and prosopagnosia. Journal of Neuropsychology, 2(1):79–97.

Hsiao, J. H.-w. and Cottrell, G. (2008). Two fixations suffice in face recognition. Psychological Science, 19(10):998–1006.

Jones, R. and Tranel, D. (2001). Severe developmental prosopagnosia in a child with superior intellect. Journal of Clinical and Experimental Neuropsychology, 23(3):265–273.

Peterson, M. F. and Eckstein, M. P. (2012). Looking just below the eyes is optimal across face recognition tasks. Proceedings of the National Academy of Sciences, 109(48):E3314–E3323.

Robertson, D. J., Noyes, E., Dowsett, A. J., Jenkins, R., and Burton, A. M. (2016). Face recognition by metropolitan police super-recognisers. PloS one, 11(2):e0150036.

Russell, R., Duchaine, B., and Nakayama, K. (2009). Super-recognizers: People with extraordinary face recognition ability. Psychonomic bulletin & review, 16(2):252–257.

Sergent, J. and PONCET, M. (1990). From covert to overt recognition of faces in a prosopagnosic patient. Brain, 113(4):989–1004.

Thomaz, C. E., Amaral, V., Giraldi, G. A., Gillies, D. F., and Rueckert, D. (2017). Is human face processing a feature-or pattern-based task? evidence using a unified computational method driven by eye movements. arXiv preprint arXiv:1709.01182.

Thomaz, C. E., Kitani, E. C., and Gillies, D. F. (2006). A maximum uncertainty ldabased approach for limited sample size problems-with application to face recognition. Journal of the Brazilian Computer Society, 12(2):7–18.

Volke, H.-J., Dettmar, P., Richter, P., Rudolf, M., and Buhss, U. (2002). On-coupling and off-coupling of neocortical areas in chess experts and novices as revealed by evoked eeg coherence measures and factor-based topological analysis–a pilot study. Journal of Psychophysiology, 16(1):23.

White, D., Phillips, P. J., Hahn, C. A., Hill, M., and O’Toole, A. J. (2015). Perceptual expertise in forensic facial image comparison. Proc. R. Soc. B, 282(1814):20151292.

VARELA, Victor P. L.; RIBEIRO, Estela; ORONA, Pedro A. S. S.; THOMAZ, Carlos E.. Eye movements and human face perception: An holistic analysis and proficiency classification based on frontal 2D face images. In: ENCONTRO NACIONAL DE INTELIGÊNCIA ARTIFICIAL E COMPUTACIONAL (ENIAC), 15. , 2018, São Paulo. Anais [...]. Porto Alegre: Sociedade Brasileira de Computação, 2018 . p. 48-57. ISSN 2763-9061. DOI: https://doi.org/10.5753/eniac.2018.4403.