Evaluating the Effectiveness of Visual Representations of SHAP Values Toward Explainable Artificial Intelligence - Moreira Cunha, Bianca; Diniz Junqueira Barbosa, Simone (Anais do Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais (IHC); 2024: Anais do XXIII Simpósio Brasileiro sobre Fatores Humanos em Sistemas Computacionais, 2024)
The employment of Machine Learning (ML) and Deep Learning models across diverse domains has grown exponentially in recent years. These models undertake critical tasks spanning medical diagnoses, criminal sentencing, and loan approvals. Consequently, there is a need for these models to be interpretable, enabling users to grasp the rationale behind predictions and engendering trust. Equally vital is the capability of developers to pinpoint and rectify any erroneous behaviors. In this context emerges the field of Explainable AI (XAI), which aims to develop methods to make ML models more interpretable while maintaining their performance level. Various methods have been proposed, many leveraging visual explanations to elucidate model behavior. However, a notable gap remains: a lack of rigorous assessment regarding the effectiveness of these explanations in enhancing interpretability. In this paper, we evaluate whether the SHAP method, widely used in the XAI field, achieves its objective of making a model more interpretable. We conducted a study based on the concept of simulatability. We asked participants who have mathematical or statistical backgrounds and intermediate to advanced knowledge of machine learning and model interpretability to classify some given instances of a dataset, presented first without the explanations provided by the SHAP method and then with them. The goal was to assess whether the explanation made the participants better at classifying the instances. Our findings reveal that the visualizations can be confusing even for users with a mathematical background. Additionally, we argue that there is a need for XAI researchers to work collaboratively with visual analytics experts to develop these visualizations, as well as test the visualizations with users of various backgrounds.