Exploring Computer Vision and AI to Assist Individuals on the Autism Spectrum with Facial Expression and Emotion Recognition

• Beatriz Barreto Marreiros Barbosa iCEV
• Carlos Futino Barreto iCEV
• Cristovam Paulo de Brito Rocha iCEV
• Francisco Luciani de Miranda Vieira iCEV
• Lílian Gabriella Castelo Branco Alves de Sousa iCEV

DOI: https://doi.org/10.5753/eries.2024.243877

Abstract

Autism Spectrum Disorder (ASD) encompasses developmental disorders that affect social interaction and communication. This research investigates the use of artificial intelligence (AI), machine learning, and computer vision to assist individuals with ASD in recognizing facial expressions and emotions. Given the rising prevalence of ASD, which affected 1 in 36 children in 2020, developing innovative tools is essential. This study combines a literature review with practical implementation, utilizing the DeepFace and OpenCV libraries in Python, to create an accessible and intuitive tool aimed at healthcare professionals, family members, and individuals on the spectrum. The tool allows for the training of social skills anywhere and anytime, stimulating the interest of people with ASD in developing their social competencies through current technologies. This approach represents a significant advancement in using advanced technologies to address the challenges associated with ASD, highlighting the importance of continued exploration and refinement of these solutions.

References

Adams, J. B., Edelson, S. M., Grandin, T., and Rimland, B. (2008). Advice for parents of young autistic children (2008, revised).

American Psychological Association (2021). Nonverbal communication.

Bandura, A. et al. (1986). Social foundations of thought and action. Englewood Cliffs, NJ, 1986(23-28):2.

Bernard-Opitz, V., Sriram, N., and Nakhoda-Sapuan, S. (2001). Enhancing social problem solving in children with autism and normal children through computer-assisted instruction. Journal of Autism and Developmental Disorders, 31(4):377–384.

Contreras-Ortiz, M. S., Marrugo, P. P., and Ribon, J. C. R. (2023). E-learning ecosystems for people with autism spectrum disorder: A systematic review. IEEE Access.

Durkin, K. and Conti-Ramsden, G. (2007). Turn off or turn on? technology, television, and language in the home. Journal of Child Psychology and Psychiatry, 48(10):1058–1065.

Ekman, P. (2003). Emotions Revealed: Recognizing Faces and Feelings to Improve Communication and Emotional Life. Henry Holt and Company.

el Kaliouby, R. and Fernandez, C. C. (2020). Girl Decoded: A Scientist’s Quest to Reclaim Our Humanity by Bringing Emotional Intelligence to Technology. Currency.

Goodwin, M. S., Intille, S. S., Albinali, F., and Velicer, W. M. (2008). Enhancing social communication through aac systems for children with autism spectrum disorders. Autism, 12(6):583–600.

Happy, S. L., Dantcheva, A., and Dugelay, J. L. (2019). Recognizing facial expressions in the wild using deep learning techniques. Journal of Computer Vision, 127(2):314–327.

Lumen Learning (2024). Introduction to sociology: Socialization. Accessed: 2024-06-04.

Maenner, M. J. (2021). Prevalence and characteristics of autism spectrum disorder among children aged 8 years—autism and developmental disabilities monitoring network, 11 sites, united states, 2018. MMWR, 70.

McQuail, D. (2001). Introdução à Teoria da Comunicação. Loyola.

OLIVEIRA, L. A. d. (2019). Aproximações entre habilidades sociais e gamificação em crianças com espectro autista: um estudo de levantamento bibliográfico.

Picard, R. W. (1997). Affective Computing. MIT Press.

Pressbooks (2024). Introduction to sociology: Socialization. Accessed: 2024-06-04.

Scassellati, B., Admoni, H., and Mataric, M. J. (2012). Robots for use in autism research. Annual Review of Biomedical Engineering, 14:275–294.

Schneider, B. H. (1993). Children’s social competence in context: The contributions of family, school and culture, volume 10. Psychology Press.

Silva, A. B. B. (2012). Mundo Singular: Entenda o autismo. Editora Fontanar.

Stravogiannis, A. L. (2020). Autismo: Um olhar por inteiro. Literare Books International, São Paulo.

Thomas Malone, D. R. e. R. L. (2023). Machine learning, explained. [link]. Acessado em: 2024-06-04.

Umberson, D., Chen, M. D., House, J. S., Hopkins, K., and Slaten, E. (1996). The effect of social relationships on psychological well-being: Are men and women really so different? American sociological review, pages 837–857.

Volkmar, F. R. (2013). Autism spectrum disorders. In Volkmar, F. R., editor, Encyclopedia of Autism Spectrum Disorders, pages 345–349. Springer.

Wainer, A. L., Ferrari, E., Dautenhahn, K., Robins, B., and Amirabdollahian, F. (2014). The effectiveness of using a humanoid robot to teach social skills to children with autism spectrum disorder. International Journal of Social Robotics, 6(4):423–435.

Zeng, Z., Pantic, M., Roisman, G. I., and Huang, T. S. (2021). A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(1):39–58.

Zilbovicius, M., Meresse, I., and Boddaert, N. (2006). Autismo: neuroimagem. Rev. Bras. Psiquiatr., 28:s21–s28.