Avaliação da interpretabilidade de modelos por meio da clusterização de explicações no contexto da predição de evasão no ensino superior
Resumo
Este trabalho propõe investigar aspectos de interpretabilidade de modelos no contexto da mineração de dados educacionais, especificamente para o problema da evasão no ensino superior. Modelos de predição são treinados e então explicados com LIME. Explicações são analisadas utilizando aprendizado não supervisionado, e é proposto um método de predição baseado em explicações centrais. O uso combinado das predições do modelo com as predições pelas explicações permite analisar aspectos de desempenho e qualidade das explicações. Apresenta-se uma métrica de interpretabilidade. Resultados indicam que modelos com performance de desempenho similar podem apresentar diferentes características quanto a métricas de interpretabilidade.
Referências
Al-Jallad, N., Ning, X., Khairalla, M., and Al-Qaness, M. (2019). Rule mining models for predicting dropout/ stopout and switcher at college using satisfaction and ses features. International Journal of Management in Education, 13(2):97–118.
Alharbi, B. (2022). Back to basics: An interpretable multi-class grade prediction framework. Arabian Journal for Science and Engineering, 47(2):2171–2186.
Alwarthan, S., Aslam, N., and Khan, I. U. (2022). An explainable model for identifying at-risk student at higher education. IEEE Access, 10:107649–107668.
Bakhshinategh, B., Zaiane, O. R., ElAtia, S., and Ipperciel, D. (2018). Educational data mining applications and tasks: A survey of the last 10 years. Education and Information Technologies, 23(1):537–553.
Burkart, N. and Huber, M. F. (2021). A survey on the explainability of supervised machine learning. Journal of Artificial Intelligence Research, 70:245–317.
Carvalho, D. V., Pereira, E. M., and Cardoso, J. S. (2019). Machine learning interpretability: A survey on methods and metrics. Electronics, 8(8):832.
Kumar, P. and Sharma, M. (2020). Predicting academic performance of international students using machine learning techniques and human interpretable explanations using lime—case study of an indian university. Advances in Intelligent Systems and Computing, 1087:289–303.
Lundberg, S. M. and Lee, S.-I. (2017). A unified approach to interpreting model predictions. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R., editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological review, 63(2):81.
Molnar, C. (2022). Interpretable Machine Learning: A Guide for Making Black Box Models Explainable. 2 edition.
Pei, B. and Xing, W. (2022). An interpretable pipeline for identifying at-risk students. Journal of Educational Computing Research, 60(2):380–405.
Ribeiro, M. T., Singh, S., and Guestrin, C. (2016). “why should i trust you?” explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135–1144.
Romero, C. and Ventura, S. (2010). Educational data mining: A review of the state of the art. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 40(6):601–618.
Vultureanu-Albisi, A. and Badica, C. (2021). Improving students’ performance by interpretable explanations using ensemble tree-based approaches. In Proceedings of SACI 2021 IEEE 15th International Symposium on Applied Computational Intelligence and Informatics, Proceedings, pages 215–220.
Xiao, W., Ji, P., and Hu, J. (2022). A survey on educational data mining methods used for predicting students’ performance. Engineering Reports, 4(5):e12482.
Alharbi, B. (2022). Back to basics: An interpretable multi-class grade prediction framework. Arabian Journal for Science and Engineering, 47(2):2171–2186.
Alwarthan, S., Aslam, N., and Khan, I. U. (2022). An explainable model for identifying at-risk student at higher education. IEEE Access, 10:107649–107668.
Bakhshinategh, B., Zaiane, O. R., ElAtia, S., and Ipperciel, D. (2018). Educational data mining applications and tasks: A survey of the last 10 years. Education and Information Technologies, 23(1):537–553.
Burkart, N. and Huber, M. F. (2021). A survey on the explainability of supervised machine learning. Journal of Artificial Intelligence Research, 70:245–317.
Carvalho, D. V., Pereira, E. M., and Cardoso, J. S. (2019). Machine learning interpretability: A survey on methods and metrics. Electronics, 8(8):832.
Kumar, P. and Sharma, M. (2020). Predicting academic performance of international students using machine learning techniques and human interpretable explanations using lime—case study of an indian university. Advances in Intelligent Systems and Computing, 1087:289–303.
Lundberg, S. M. and Lee, S.-I. (2017). A unified approach to interpreting model predictions. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R., editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological review, 63(2):81.
Molnar, C. (2022). Interpretable Machine Learning: A Guide for Making Black Box Models Explainable. 2 edition.
Pei, B. and Xing, W. (2022). An interpretable pipeline for identifying at-risk students. Journal of Educational Computing Research, 60(2):380–405.
Ribeiro, M. T., Singh, S., and Guestrin, C. (2016). “why should i trust you?” explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135–1144.
Romero, C. and Ventura, S. (2010). Educational data mining: A review of the state of the art. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 40(6):601–618.
Vultureanu-Albisi, A. and Badica, C. (2021). Improving students’ performance by interpretable explanations using ensemble tree-based approaches. In Proceedings of SACI 2021 IEEE 15th International Symposium on Applied Computational Intelligence and Informatics, Proceedings, pages 215–220.
Xiao, W., Ji, P., and Hu, J. (2022). A survey on educational data mining methods used for predicting students’ performance. Engineering Reports, 4(5):e12482.
Publicado
06/11/2023
Como Citar
CARVALHO, Cássio S.; MATTOS, Júlio C. B.; AGUIAR, Marilton S..
Avaliação da interpretabilidade de modelos por meio da clusterização de explicações no contexto da predição de evasão no ensino superior. In: SIMPÓSIO BRASILEIRO DE INFORMÁTICA NA EDUCAÇÃO (SBIE), 34. , 2023, Passo Fundo/RS.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2023
.
p. 1191-1201.
DOI: https://doi.org/10.5753/sbie.2023.234435.
