Analysis of the Robustness of Machine Learning Algorithms on Autism Spectrum Disorder Data

• Saulo B. F. Lino Federal University of Ceará (UFC)
• Lívia A. Cruz Federal University of Ceará (UFC)
• Paulo T. Guerra Federal University of Ceará (UFC)

DOI: https://doi.org/10.5753/sbbd.2024.240567

Abstract

Autism Spectrum Disorder (ASD) is a neurological condition that affects communication, social interaction, behavior, and learning. Screening methods such as AQ and Q-CHAT have been developed to speed up the identification of autistic signs. The present work analyzes the performance of machine learning algorithms in ASD screening, {such as SVM, MLP, Logistic Regression, Naive Bayes, Random Forest, and KNN}, and the robustness of these models in the face of possible errors in the data. The algorithms are evaluated on datasets with samples based on personal characteristics and simplified questions from the AQ and Q-CHAT instruments. The experiments show good performance of the SVM, MLP, and Logistic Regression methods, but with a significant reduction in their accuracy in scenarios with errors.

Keywords: Machine Learning, Classification Algorithms, Data and Information Quality

References

Allison, C., Auyeung, B., and Baron-Cohen, S. (2012). Toward brief “red flags” for autism screening: The short autism spectrum quotient and the short quantitative checklist in 1,000 cases and 3,000 controls. Journal of the American Academy of Child & Adolescent Psychiatry, 51(2):202–212.

Allison, C., Baron-Cohen, S., Wheelwright, S., Charman, T., Richler, J., Pasco, G., and Brayne, C. (2008). The q-chat (quantitative checklist for autism in toddlers): a normally distributed quantitative measure of autistic traits at 18–24 months of age: preliminary report. Journal of autism and developmental disorders, 38:1414–1425.

APA, A. P. A. (2013). Diagnostic and Statistical Manual of Mental Disorders. American Psychiatric Association.

Artoni, A. A., Barbosa, C., and Morandini, M. (2022). Autism spectrum disorder diagnosis assistance using machine learning. Revista de Informática Teórica e Aplicada, 29(3):36–53.

Baron-Cohen, S., Wheelwright, S., Skinner, R., Martin, J., and Clubley, E. (2001). The autism-spectrum quotient (aq): Evidence from asperger syndrome/high-functioning autism, malesand females, scientists and mathematicians. Journal of autism and developmental disorders, 31:5–17.

Ferreira, R. d. S. (2010). Autism testing: Uma ferramenta móvel no auxílio ao pré-diagnóstico do autismo. In Anais do XXII Conferência Internacional sobre Informática na Educação. Fortaleza, Ceará-Brasil: Nuevas Ideas en Informática Educativa, volume 13, pages 178–187.

Fitzgerald, M. (2017). The clinical gestalts of autism: Over 40 years of clinical experience with autism. In Fitzgerald, M. and Yip, J., editors, Autism, chapter 2. IntechOpen.

Garg, A., Parashar, A., Barman, D., Jain, S., Singhal, D., Masud, M., and Abouhawwash, M. (2022). Autism spectrum disorder prediction by an explainable deep learning approach. Computers, Materials & Continua, 71(1):1459–1471.

Hossain, M. D., Kabir, M. A., Anwar, A., and Islam, M. Z. (2021). Detecting autism spectrum disorder using machine learning techniques: An experimental analysis on toddler, child, adolescent and adult datasets. Health Information Science and Systems, 9:1–13.

Kleinman, J. M., Robins, D. L., Ventola, P. E., Pandey, J., Boorstein, H. C., Esser, E. L., Wilson, L. B., Rosenthal, M. A., Sutera, S., Verbalis, A. D., Barton, M., Hodgson, S., Green, J., Dumont-Mathieu, T., Volkmar, F., Chawarska, K., Klin, A., and Fein, D. (2008). The modified checklist for autism in toddlers: A follow-up study investigating the early detection of autism spectrum disorders. Journal of Autism and Developmental Disorders, 38(5):827–839.

Lord, C., Risi, S., Lambrecht, L., Cook, E. H., Leventhal, B. L., DiLavore, P. C., Pickles, A., and Rutter, M. (2000). The autism diagnostic observation schedule—generic: A standard measure of social and communication deficits associated with the spectrum of autism. Journal of autism and developmental disorders, 30:205–223.

Lord, C., Rutter, M., and Le Couteur, A. (1994). Autism diagnostic interview-revised: a revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of autism and developmental disorders, 24(5):659–685.

Lundberg, S. M. and Lee, S.-I. (2017). A unified approach to interpreting model predictions. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R., editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.

Ribeiro, M. T., Singh, S., and Guestrin, C. (2016). ”why should i trust you?”explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135–1144.

Thabtah, F. (2017). ASDTests: A mobile app for ASD screening. Disponível em: [link]. Acesso em: 10 de maio de 2024.

Thabtah, F., Abdelhamid, N., and Peebles, D. (2019). A machine learning autism classification based on logistic regression analysis. Health Information Science and Systems, 7:1–11.

Thabtah, F., Kamalov, F., and Rajab, K. (2018). A new computational intelligence approach to detect autistic features for autism screening. International journal of medical informatics, 117:112–124.

Thabtah, F. and Peebles, D. (2020). A new machine learning model based on induction of rules for autism detection. Health informatics journal, 26(1):264–286.