Evaluating the explainability of BrainGNN on prediction of ASD diagnosis

• Matheo Angelo Pereira Dantas USP
• André Carlos Ponce de Leon Ferreira de Carvalho USP

DOI: https://doi.org/10.5753/eniac.2025.11771

Abstract

Due to the absence of a biological marker for Autism Spectrum Disorder (ASD), most of recent research attempts to uncover the neurological patterns of ASD using Deep Learning, and these patterns, hidden in the latent feature space of neural networks, have to be interpreted with the use of Explainable AI. However, although many of the models proposed for the problem report results of explainability, they are not evaluated with any metric, so their reliability is unknown. The objective of this paper is to propose an evaluation framework to fill this gap, and here, we focus on a detailed analysis of a well-known model from the literature, BrainGNN. We trained BrainGNN in varying hyperparameter settings that influence explainability and analyzed our findings for each case.

References

American Psychiatric Association (2013). Diagnostic and Statistical Manual of Mental Disorders, 5th Edition. American Psychiatric Publishing, Washington, D.C., 5 edition.

Brazil (2012). Law no. 12,764, of december 27, 2012. establishes the national policy for the protection of the rights of persons with autism spectrum disorder. Known as "Berenice Piana Law" or "Brazilian Autism Law". It recognizes autism spectrum disorder as a disability for all legal purposes in Brazil.

Bronstein, M. M., Bruna, J., LeCun, Y., Szlam, A., and Vandergheynst, P. (2017). Geometric deep learning: going beyond euclidean data. IEEE Signal Processing Magazine, 34(4):18–42.

Cangea, C., Veličković, P., Jovanović, N., Kipf, T., and Liò, P. (2018). Towards sparse hierarchical graph classifiers. arXiv preprint arXiv:1811.01287.

Dai, E., Zhao, T., Zhu, H., Xu, J., Guo, Z., Liu, H., Tang, J., and Wang, S. (2024). A comprehensive survey on trustworthy graph neural networks: Privacy, robustness, fairness, and explainability. Machine Intelligence Research, 21(6):1011–1061.

Gao, H. and Ji, S. (2019). Graph u-nets. In international conference on machine learning, pages 2083–2092. PMLR.

Huang, Z.-A., Zhu, Z., Yau, C. H., and Tan, K. C. (2020). Identifying autism spectrum disorder from resting-state fmri using deep belief network. IEEE Transactions on neural networks and learning systems, 32(7):2847–2861.

Li, X., Zhou, Y., Dvornek, N., Zhang, M., Gao, S., Zhuang, J., Scheinost, D., Staib, L. H., Ventola, P., and Duncan, J. S. (2021). Braingnn: Interpretable brain graph neural network for fmri analysis. Medical Image Analysis, 74:102233.

Lin, Y., Yang, J., and Hu, W. (2022). Denoising fmri message on population graph for multi-site disease prediction. In International Conference on Neural Information Processing, pages 660–671. Springer.

Luo, X., Wu, J., Yang, J., Xue, S., Beheshti, A., Sheng, Q. Z., McAlpine, D., Sowman, P., Giral, A., and Yu, P. S. (2024). Graph neural networks for brain graph learning: a survey. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, pages 8170–8178.

Schlichtkrull, M., Kipf, T. N., Bloem, P., Van Den Berg, R., Titov, I., and Welling, M. (2018). Modeling relational data with graph convolutional networks. In The semantic web: 15th international conference, ESWC 2018, Heraklion, Crete, Greece, June 3–7, 2018, proceedings 15, pages 593–607. Springer.

Smith, S. M., Jenkinson, M., Woolrich, M. W., Beckmann, C. F., Behrens, T. E. J., Johansen-Berg, H., Bannister, P. R., De Luca, M., Drobnjak, I., Flitney, D. E., Niazy, R. K., Saunders, J., Vickers, J., Zhang, Y., De Stefano, N., Brady, J. M., and Matthews, P. M. (2004). Advances in functional and structural mr image analysis and implementation as fsl. NeuroImage, 23:S208–S219.

Wang, H., Fu, T., Du, Y., Gao, W., Huang, K., Liu, Z., Chandak, P., Liu, S., Van Katwyk, P., Deac, A., et al. (2023). Scientific discovery in the age of artificial intelligence. Nature, 620(7972):47–60.

Wu, T., Ren, H., Li, P., and Leskovec, J. (2020). Graph information bottleneck. Advances in Neural Information Processing Systems, 33:20437–20448.

Yan, J., Chen, Y., Xiao, Z., Zhang, S., Jiang, M., Wang, T., Zhang, T., Lv, J., Becker, B., Zhang, R., et al. (2022). Modeling spatio-temporal patterns of holistic functional brain networks via multi-head guided attention graph neural networks (multi-head gagnns). Medical Image Analysis, 80:102518.

Yang, Y., Cui, H., and Yang, C. (2023). Ptgb: Pre-train graph neural networks for brain network analysis. arXiv preprint arXiv:2305.14376.

Yuan, H., Yu, H., Gui, S., and Ji, S. (2022). Explainability in graph neural networks: A taxonomic survey. IEEE transactions on pattern analysis and machine intelligence, 45(5):5782–5799.

Yuan, H., Yu, H., Wang, J., Li, K., and Ji, S. (2021). On explainability of graph neural networks via subgraph explorations. In International conference on machine learning, pages 12241–12252. PMLR.

Zhang, S. and Chiang-shan, R. L. (2012). Functional connectivity mapping of the human precuneus by resting state fmri. Neuroimage, 59(4):3548–3562.

Zhang, S., Yang, J., Zhang, Y., Zhong, J., Hu, W., Li, C., and Jiang, J. (2023a). The combination of a graph neural network technique and brain imaging to diagnose neurological disorders: A review and outlook. Brain Sciences, 13(10):1462.

Zhang, S., Yang, J., Zhang, Y., Zhong, J., Hu, W., Li, C., and Jiang, J. (2023b). The combination of a graph neural network technique and brain imaging to diagnose neurological disorders: A review and outlook. Brain Sciences, 13(10):1462.

Zheng, K., Yu, S., Chen, L., Dang, L., and Chen, B. (2024a). Bpi-gnn: Interpretable brain network-based psychiatric diagnosis and subtyping. NeuroImage, 292:120594.

Zheng, K., Yu, S., Li, B., Jenssen, R., and Chen, B. (2024b). Brainib: Interpretable brain network-based psychiatric diagnosis with graph information bottleneck. IEEE Transactions on Neural Networks and Learning Systems.